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Executive Summary 



California's Pupil Proficiency Law (AB 3408 as amended by AB 65) requires 
each local school district to adopt standc^rds of proficiency in the basic skills 
of reading comprehension, writing, and computation. Students are to be assessed 
in grades four-six, seven-nine, ten, and eleven. Students not making sufficient 
progress towards meeting the district's standards are to be provided with reme- 
dial programs, and school personnel are required to hold conferences with par- 
ents to inform them of their child's weaknesses in basic skills. After June of 
1980, no student may be granted a high school diploma unless he or she has met 
district proficiency standards and completed the course requirements set forth 
by the school district. The law grants broad discretion to school districts in 
determining their basic skills assessment and instructional procedures. The 
State Board and State Department of Education are precluded from either promul- 
gating a statewide proficiency test or conducting monitoring or compliance re- 
views of local procedures. Instead, the role of the Department has been limited 
to provision of technical assistance and training to assist districts in meeting 
the requirements of the law. 

Purpose of the Study 

This study was designed to examine the progress of local school districts 
in implementing the Pupil Proficiency Law. In addition to reviewing overall 
implementation of the law, the study report also attempts to point out problem 
areas which may require policy clarification, redirection of state technical 
assistance, or both. The study was initiated at the request of the Assembly 
Education Committee, Subcommittee on Educational Reform, and the Assembly Ways 
and Means Committee, Subcommittee on Education. It builds on the findings of 
a similar study conducted by the Department during 1978-79. 

Study Procedures 

The study utilized a representative sample of 155 school districts. Ques- 
tionnaires mailed to district offices were used to elicit infoLiVation on district 
test8> enumerative data related to the proficiency law, and specific implementa- 
tion issues. In addition, in-depth case studies were conducted in 15 districts. 
The case studies involved interviews with district personn^^l, principals, and 
teachers. Student reactions to proficiency testing were gathere<^ through a ques- 
tionnaire administered at each high school in the case study sample. A complete 
description of study methods and procedures is included in Chapter 1. 

Major Findings 

Major study findings are organized into three areas: district tests, 
curriculum and instruction, and effects on students. 



Proficiency Assessment in California; A Status Report on Implementation of 
the Requirements of AB 3408/76 and AB 65/^7 . Sacramento: California State 
Department of Education, 1979. 
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District Teata 



A large majority (78 percent) of school districts have developed thoii own 
proficiency tests, rather than purchasing commercially developed tests. This 
approach seems to have engendered under^itanding and ownership of the tests among 
teachers* However, because of a lack of expertise in test construction among 
local district staffs, many of the locally developed tests displayed flawj, such 
as inadequate directions for test takers, loor layout, lack of teat item speci~ 
fications, and failure to review for bias. Many of these tests could benefit 
from careful review and refinement. 

Content analyses of the skills included on district tests revealed a rela- 
tively consistent view of basic skills across districts. Very few districts 
in the Department's sample chose to assess skills that were very simple or un- 
usually complex. The most striking variation was in the choice of whether to 
assess "school skills," "life skills," or some combination of the two. Many 
districts (47 percent) assessed a combination of "school skills" and "life 
skills." Thirty-nine percent of districts chose to emphasize primarily "school 
skills," while 15 percent emphasized "life skills." 

Given the lack o^f comparability of district tests, it is technically un- 
feasible to pinpoint the relative "difficulty" of these tests. Instead, the 
study used a group of curriculum content specialists to examine the complexity 
of skills assessed by districts. After reviewing a sample of tests, the math- 
ematics and writing specialists concluded that the tests consistently reflected 
minimum basic skills. The reading specialists were unable to reach any firm 
conclusions on the relative complexity of reading comprehension skills assessed. 
Most districts seem to be assessing the same types of basic skills. While the 
data revealed a few subsections of tests that were unusually simple or complex, 
the complexity of skills assessed on district proficiency tests does not appear 
to vary a great deal. 

Curriculum, Instruction, and Parent Conferences 

Administrators in 42 percent of the high school and unified school dis- 
tricts reported that the Proficiency Law was having a "significant" effect on 
curriculum and instruction. Another 56 percent of these districts reported at 
least a "marginal" effect. In addition, more than 90 percent of the high school 
and unified school districts reported that more time was now being devoted to 
basic skills instruction than in the past. The data also suggested that dis- 
tricts are making serious efforts to link proficiency objectives to their local 
curriculum and instructional program. 

Some of the study data addre?i?^d the question of whether basic skills were 
"taking over" the curriculum and instruction in high schools to the detriment of 
more advanced courses and other subject areas. First, it was apparent that many 
more students than miglit have been expected (30-50 percent of tenth graders) 
were failing at least one portion of the proficiency test, and that these stu- 
dents were being enrolled in one or more remedial classes. To the extent these 
remedial classes consume student instructional time, it is likely that opportu- 
nities to take advanced or elective courses have been limited. Moreover, the 
English and mathematics teachers interviewed indicated that responsibility for 
planning and conducting basic skills and remedial instruction was falling dis- 
proportionately on high school English and mathematics departments. This, the 
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teacherH feU, unquuationab ly had limited tha ftvaiUhility of advancud dourauw 
in Engliah and mflthemntica. The paradox inherent in thia iaau© ia how much 
atudentfi who are deficient in baaic aUilla could benefit from more advanced 
couraea, whether in Engliah, mathematica, or other aubject areaa. Hut, if 
Btudenta are not cliallenged to aapiro to learn higher-'level content and akiUs, 
there is a danger that the high achool curricuium could become trivialized. 
This iasue certainly merita further atudy. 

The Proficiency Law requires schools to request parents of atudenta failing 
proficiency tests to appear for conferences to discuss student progress and re- 
view the kinds of remedial programs the school will be providing. This provi"- 
sion was intended to foster a close working relationship between home and school 
in helping the student to improve his or her performance. Study data suggest 
that large numbers of parents of high school students are not appearing for 
conferences. About 25 percent of the sample districts reported that less than 
half of the parents of failing students attended the conferences. Eleven per- 
cent of the districts reported that less than 25 percent of the parents came to 
the conferences. With little more than one year left before diplomas are to be 
withheld if students fail to meet proficiency standards, this problem seems se- 
rious. Concerted efforts seem to be warranted to ensure that parents are aware 
of proficiency requirements and are urged to respond to requests to appear for 
conferences. 

Most districts are using a variety of approaches to provide remedial pro- 
grams for students who fail proficiency tests. Virtually all districts reported 
using in-class remedial work, while about two-thirds reported using tutorial 
programs. About 60 percent of the districts were using special **pull out" pro- 
grams. Another 40 percent of the districts were planning to use summer school. 
Districts reported a number of problems in organizing remedial programs. First, 
many high school teachers were inadequately trained to teach remedial basic 
skills, and there was little evidence that extensive in-service training had been 
made available to upgrade the teachers' skills. Second, while many districts re- 
sponding indicated that "basic skills" curricular materials had been developed, 
few reported providing in-service training for teachers in effective "basic 
skills" instructional practices. Finally, student absenteeism emerged as a major 
problem, especially in the larger urban high schools. Many teachers commented 
that in addition to their responsibilities to provide programs which would moti- 
vate students to learn basic skills, they faced a prior task — motivating students 
to attend school in the first place. 

Effects on Students 

In each of the case study schools, a class of eleventh grade students was 
surveyed in order to learn what students knew about proficiency requirements and 
to get their general reactions to the tests. This sample is not a representa- 
t ive cross-section, but student awareness was none the less encouraging. Almost 
all (96 percent) of the students surveyed knew that they had to pass proficiency 
tests in order to graduate from high school. Almost 90 percent of the students 
responding had taken one or more proficiency tests, and three-quarters of these 
students felt that the tests were set at "about the right degree of difficulty." 
The remaining 25 percent of students were evenly divided as to whether the test 
was "too hard" or "too easy." 
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Introduction 



Thd pupil proflcl^noy rt^qulr^m^uiti Hvui h^^mm Uw lii t\\^ t^li or PJ/o 
(AB 3408) nud wara Ut;iir modified in 1977 (AH 65). Thiay raqulra AUultinU In 
GallfornlA schoals to damonatrAt^i protioioncy lu b«iilc tikllUi as meAaur^d 
by Ch« looiiUy iidppt;«d »tandardfli prior to r«Q«iving a high school diploma. 
After Jun© of 1980, atudants will be expact^d to maet dUtrict profidduoy ra- 
(tMir«m«nti in reading Qomprehiin«ioni writ;ingi md i;iumpuUt;ion iu mtdltion t;o 
meeting locel oourte of etudy graduation requirementii* The oUse of 1981 ia 
the firet oUes to be effected by the dlplome eenction« 

Proficiency requirementa direct local dlatricte toj (a) adopt proficiency 
■tandarde in reading comprehensloni vritingi end computation; (b) aelect or 
develop meaaureu or procedurea to naaeao atudent proficiency and aet paaaing 
acoree; (c) adminiateri acore. and report proficiency teat reaulta; (d) hold 
conferencee and remediate atudenta not making aufficient progreaa; (e) provide 
opportunities for reaaaeaamont; and (f) deny diplomaa to atudenta who do not 
demonstrate proficiency in the baaic akilla* 

The State Board of Educationi through the State Department of Education, la 
required to: (a) make available a framework for proficiency aaaeaament which 
includea aample aaaeaament itema; and (b) provide information on the atatua of 
district implementation activitiea to the Legialature. The State Department of 
Education la not required to collect any type of monitoring or compliance data 
from local school diatrlcta* 

The iaauea examined in this study were identified jointly by representa- 
tives of the Department of Education and the Legislature. Mounting concern 
dbout the possibility of legal challenges to California's pupil proficiency law 
influenced, In part, the issues that were investigated. 

In testimony before the Assembly Subcommittee on Educational Reform on 
December 10, 1979^ Department representatives outlined the legal issues the 
proficiency law was likely to be challenged on. Based on analysis of the legal 
issues, the Department noted that future court holdings related to proficiency 
testing would likely depend heavily on the decision of the U.S. District Court 
for the Middle District of Florida in Debra P. v. Turlington . 

In summary, the court held: (I) Florida's competency testing program did 
not give all students adequate notice of the fact that they would have to pass 
a competency test in order to graduate; and (2) the competency testing program 
carried forward the effects of prior racial discrimination in violation of the 
du^ process and equal protection clauses of the Fourteenth Amendment of the 
U.S. Constitution, Title VI of the Civil Rights Act of 1964, and the Equal Edu- 
cational Opportunities Act of 1974. As a remedy/ the court enjoined Florida 
from using the test as a diploma requirement for four years — until the 1982-83 
school year. The court did not, however, enjoin use of the competency test dur- 
ing this four-year period for assessing the effects of instruction. In addi- 
tion to the notice and discrimination Issues, the Department's analysis also 



No. 78-892 Clv.-T-H (M.D.Fla. decision 7/12/79), 48 L.W. 2058 (7/24/79). 




iii(:Miittii| 4n 4^uva iiu<arti4t in ilit*i^i^t imp liiiudntMUou prot ic luiuvy 4««d«rt'- 
muiUf III Apt il of ly^Mi tha aubcommictua raqMfcititaU thrtt th^ UMpartm^ut aondui^l- 
4 aurvey ot all I0C4I aclu^ol diatrictti to g4ti)tir information on ^iititrict progrtiay 
m impi 0iu^nn im Hi^ Uiw 4iul to i^ldnUfy ptobUm 4rM44 and uts^id^i ii)r (Mihuiiial 
4«4irtt4nce. Thii rttoily report w44 pnhli«hail in Jni\tJ, l)4t4 from fh4t wtuily 

4nil tha cnruMU- «tndy wi^rti n^ail to; 

!• Provula the l^dginhitnro with npil4ttu| Information on implomontat ion 
4rti/itiii« tor information and Uariwion making, 

2. l^rovUIti tlui DopartmMnt with informati<ni on technical aHuiHtauco nutuU 
of local (liHtrictH. 

Thti cnrrwnt Htndy followH up on IrtMui^M rai«o<l in tho Jwn^, 1979» ruport ' 
and oxaminoN : 

L The contont and variability of difltrict tentn 

2, Notice to atudontH and parents about proficioncy aaaeaament 

3, Relationship between district teata and local curriculum and 
inatruc tion 

4, Procedurea Cor conferences and the nature of remedial progr/uua 

5, Effects on students, including minority atudents and drop-outs 

6, Extent to which differential standards policies for special education 
students have been adopted 
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See California State Department of Education, "Implementation of California's 
Pupil Proficiency Law," Testimony Before the Assembly Education Committee's 
Subcommittee on Educational Reform, December 10, 1979. 
3 . . 

Proficiency Assessment in California: A Status Report on Implementation of 
the Requirements of AB 3408/76 and AB 65/77 > Sacramento: California State 
Department of Education, 1979. 
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Chapter 1 

Methodology 



This 1980 study of proficiency assessment is a descriptive account of dis- 
trict implementation activities. Information on these activities was collected 
in three ways: 

!• A content analysis of district tests to describe their form and content 
( Measures Substudy ) 

2. A survey to gather data on notification and the use of test results, 
experiences with conferences and supplementary instruction, effects 
on special populations, and the impact of proficiency assessment on 
curriculum and instruction ( Implementation Substudy ) 

3. A series of structured interviews to gather more detailed data on 
questions raised in the Implementation Substudy ( Case Studies ) 

The remainder of this chapter provides a description of the design of each 
subr^tudy • 



Measures Substudy 

The Measures Substudy, which began in the spring of 1979, examined the con- 
tent and form of a sample of district tests and gathered survey data about test 
development processes and policies. 

^* Sample : A stratified sample of 144 districts was drawn using four demo- 
graphic variables to stratify: district size (based on enrollment); district 
type (unified, elementary, or high school status); district location , (ur- 
ban, suburban, or rural); and district socioeconomic status (SES) ( percent 
of students in families receiving Aid to Families with Dependent Children 
(AFDC)). Table 1 describes the sample. 

The six largest districts in the state, representing approximately 21 
percent of the total student population, were included in the sample. 

B. Instrumentation : A questionnaire was mailed to 144 district superinten- 
dents on May 21, 1979. Directions specified that the questionnaire should 
be completed by the person most knowledgeable about proficiency assessment. 
Questions were designed to gather information on the following issues: 

1. How the test was developed 

2. How passing scores were set 

3. How test results were used 

4. Percent of the class of 1981 that passed the initial test 



^Taken from the California Assessment Program. 
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TABLE 1 



MEASURES SUB STUDY SAMPLE 



Demographic variables 


Districts 


Rate of 
return 


Number 

in 
sample 


Percent 
of total 


Number 
re- 
sponding 


Type of district 










Unified 


53 


36% 


48 


91% 


Elementary 


57 


40 


44 


77 


High school 


34 


24 


26 


76 


Community size 










Urban 


24 


17% 


21 


87% 


Suburban 


62 


43 


57 


92 


Rural 


58 


40 


40 


Oy 


District size 










Low a.d.a.* 


56 


39% 


39 


70% 


Medium a«d.a»* 


50 


35 


43 


86 


High a.d.a.* 


38 


26 


36 


95 


Socioeconomic status 










Low percent AFDC** 


50 


35% 


38 


76% 


Medium percent AFDC** 


37 


25 


29 


78 


High percent AFDC** 


57 


40 


51 


89 


Total responding 


N=144 




118 


82% 



*Average daily attendance 
**Aid to Families with Dependent Children 



Districts were requested to submit copies of their test(s) with test 
specif icationsy field test information, and other supporting data. The 
letter requesting tests assured districts that the Department would keep 
tests and supporting materials confidential. 

Respondents : The last column of Table I shows the rate of return to the 
Measures Survey, by demographic variables. Although 118 questionnaires 
were returned, only 65 districts submitted copies of their tests. Fewer 
districts supplied supporting documentation. 

To increase the response rate, follow-*up phone calls were made to 
the district proficiency assessment coordinator, and a second copy of the 
questionnaire was sent. Although both requests produced 118 responses, 
districts which withheld their tests cited the following reasons: 

1. Secure tests : A few districts chose not to send sample tests, despite 
the Department's assurance that they would be kept confidential. 

* 14 



2, Commercial tests ; Districts that used commercial tests responded that 
sample tests could be acquired from the publisher, 

3, Consortium-adopted tests ; Districts that developed their test coop- 
eratively often did not have the authority to release tests also used 
by other districts, 

4, Elementary districts ; Many elementary districts had not completed 
test development at the time the questionnaire was received, 

I 

D, Analysis : Three procedures were used to analyze the data, 

1, Questionnaire responses were totalled and are reported by percents ad- 
justed for missing data (percents of responses on each item). When 
appropriate, responses were analyzed by demographics of the sample. On 
questions where multiple responses were expected, percents total more 
than 100, 

2, A subsample of tests was reviewed by subject-matter specialists from 
districts and universities throughout the state. The analysis included 
investigation of; 

a. Number and type of skills being assessed 

b. Form and method of assessment 

c. Appraisal of item and test characteristics 

3, Documentation accompanying tests included test manuals, directions for 
administration, and curricular materials linked to proficiencies being 
tested. These materials were also analyzed according to; 

a. General test characteristics 

b. Characteristics of; computation proficiencies and test items; 
reading comprehension proficiencies and test items; and writing 
proficiencies and test items 

The findings from the Measures Substudy are reported in Chapter 2, 



Implementation Substudy 

The Implementation Substudy conducted during the fall of 1979 examined indi- 
vidual district implementation efforts, including; notice, test results, confer- 
ences, curriculum and instruction, remedial instruction, and effects on students 
with special needs, 

• 

A. Sample; The 144 districts selected for the Measures Substudy were included 
in the sample for the Implementation Substudy, The sample was stratified 
i along demographic variables of size, type, and SES categories. The Measures 

sample was expanded by 11 districts in order ^to ensure a proportional sample 
of high school and unified school districts. Table 2 describes the sample. 



Sample revisions are illustrated in the sampling framework in the appendix. 
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TABLE 2 



IMPLEMENTATION SUBSTUDY SAMPLE 



Demographic variables 


Districts 


Rate of 
return 


Number 

in 
sample 


Percent 
of total 


Number 
re- 
sponding 


Type 










Unified 


62 


40% 


40 


Ohm 


Elementary 


57 


37 


38 


66 


High school 


36 


23 


29 


80 


District size 










Low a • d • a • ^ 


^7 
J / 




0 c 

JJ 




Medium a«d«a«* 


60 


39 


42 




High a.d.a.* 


38 


24 


30 


79 


Socioeconomic status 










Low percent AFDC** 










(7.6% or less) 


81 


52% 


49 


60% 


High percent AFDC** 










(8.2% or more) 


74 


48 


52 


70 


Total responding 


N-155 




107 


69% 



*Average daily attendance 
**Aid to Families with Dependent Children 



B. Instrumentation : A questionnaire was developed and mailed on October 19, 
1979, to those district persons identified in the Measures Substudy as 
responsible for implementation of proficiency assessment. 

Questions were designed to gather information on the following issues: 

1. Initial impact of proficiency assessment activities on curriculum and 
instruction and activities to ensure that assessed skills are included 
in course curricula and instruction 

2. Notification of students, parents, and teachers of various proficiency 
assessment activities and the report and use of test results 

3. Provision for conferences and organization of remedial instruction 

4. Effects of proficiency assessment on special students (including 
linguistic and racial minorities) and drop-out rates 

C. Respondents : In the last column of Table 2 is the rate of return shown, by 
demographic variables. Follow-up letters to districts were sent to increase 
the response rate. 
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D» Analysis ; Survey responses were totalled and frequencies are reported by 
percents adjusted £or missing data (percents o£ responses on each item). 
When appropriate, responses were analyzed by demographics of the sample, 
VRiere multiple responses were expected, percents total more than 100, 
Findings for this part of the study are reported in Chapter 3. 

Case Studies 

The Case Studies, conducted in November-December 1979, were a series of 
structured interviews designed to gather in-depth perceptions of district 
activities and concerns regarding proficiency assessment, 

A, Sample ; A sample of 15 districts was selected for the Case Studies, 

Thirteen of these districts were included in the Implementation Substudy, 
Criteria used in the selection of case study districts were; geographic 
and demographic diversity, type of test being used for proficiency assess- 
ment, and approaches to proficiency assessment relative to other testing 
act ivities , 

B* Ins tr amenta t ion ; A detailed structured interview was used. Questions 
addressed generally the same topics covered in the Implementation Survey, 
In addition, questions solicited information on effects of proficiency 
assessment on morale, communication, classroom instruction, and student 
achievement, A short questionnaire was administered to a total of 413 
eleventh grade students in the case study high schools. The questionnaire 
covered student knowledge of the requirements, the local test, and provi- 
sions for remedial instruction* The principal of each high school visited 
selected one classroom of students to be surveyed, 

C* Respondents ; Structured interviews were conducted with the following 
individuals in each district. - 

1, District person responsible for proficiency assessment, typically an 
assistant superintendent or director of testing 

2, Principals or vice-principals of two high schools (where the district 
had more than one high school) 

3, Math and English department chairpersons in high schools visited 

4, Teacher or coordinator responsible for remedial instruction 

In addition, various other persons, including guidance counselors and 
curriculum specialists, were interviewed informally, 

D. Analysis ; Each interviewer summarized interviews for the case study dis- 
trict he or she visited. Anecdotes from these cases are used to supplement 
the report. 

Chapter 4 contains two case study accounts of district implementation 
activities. 



These two districts were selected as illustrating the particular 
processes and problems of implementing the state proficiency assessment 
mandate. Case Study 1 is a large, urban district with a high minority pop*- 
ulation. The district has considerable measurement expertise and has "gone 
beyond the law" to adopt standards in social studies and sciences. Case 
Study 2 is a small, rural district with a small minority population. This 
district has a limited number of staff working on proficiency assessment 
and limited measurement expertise. 
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Chapter ? 

District Proficiency Tests 

The Measures Substudy addressed the following major questions: 

How were the tests developed? 

What skills were included in the test? 

How were they measured? 

How were the test results used? 

A one-page questionnaire was sent to a sample of 144 school districts* 
District personnel responsible for proficiency assessment were asked to respond 
to this survey and submit copies of their proficiency tests and related material* 
Multiple responses were possible for each question (percent s may total more than 
100). A description of the methodology for this substudy appears in Chapter !• 

Test Development Procedures 

California's Pupil Proficiency Law requires that districts adopt tests that 
assess individual student performance in reading comprehension, writing, and 
computation* 

Test Development 

Table 3 includes the district responses about involvement in test devel- 
opment. Tests were more likely to be district developed than commercially de- 
veloped or purchased. Administrators report that tests were developed by 
teachers in 73 percent of the elementary districts, 80 percent of the high 
school districts, and 81 percent of the unified districts. 

TABLE 3 



WHO DEVELOPED THE PROFICIENCY TEST? 





Percent of responding districts 






Elemen- 


High 






All 


tary 


school 


Unified 


Test source 


(n=I05) 


(n=37) 


(n=26) 


(n=42) 


District-developed 










by teachers 


78% 


73% 


80% 


81% 


District-developed by 










central office staff 


36 


35 


27 


43 


District-developed 










with consultants' 










ass istance 


39 


54 


46 


21 


Custom-developed 










commercial test 


II 


5 


12 


17 


Off-the-shelf 










commercial test 


19 


5 


31 


24 



9 

is 



Analysis by demographic variables indicated that large districts were more 
likely to use central office staff assistance and customized commercial tests 
than were small or medium size districts* 

Source of Items 

Using items from item pools eases the test development process for local 
districts. Items in item pools or banks typically have been field test(3d and 
have validity and reliability estimates. In order to get some indication of the 
use of the state Sample Assessment Exercises Manual ( SAEM ) and the use of othfer 
item pools, the measures questionnaire asked districts where they obtained their 
items. Table 4 contains a summary of their responses. 



TABLE 4 

ARE DISTRICTS OBTAINING TEST ITEMS FROM ITEM POOLS? 



Percent of responding districts 



Item source 


All 


Elementary 


Secondary 


Unified 


(n-103) 


(n-36) 


(n-26) 


(a'«41) 


Yes, SAEM (SDE- 










developed items) 


40% 


42% 


38% 


37% 


Yes, public 










agency item pool 


22 


31 


23 


15 


Yes, private firm 


16 


6 


23 


22 


No, other source 


46 


50 


38 


56 



Half of the responding districta were using items from item pools. Forty 
percent of the responding districts had used items from the different volumes of 
the Sample Assessment Exercises Manual ( SAEM ) , developed by the Office of Pro- 
gram Evaluation and Research. Items from these manuals had been used in 90 per- 
cent of the districts where teachers assisted in test development and in half 
of the districts where central office staff and consultants had developed the 
proficiency tests. 

Elementary school districts used public item pools for their tests more than 
high school or unified districts did, and the elementary districts used private 
item pools much less than the other districts did. It is not clear whether this 
was because of the cost factor, availability, or a lack of test items in private 
item pools for lower grade levels. Other sources of items included exercises 
adapted from local curricula and testing materials. 

Influences on Passing Scores 

Districts used a combination of approaches to set passing scores. Two- 
thirds of the responding districts used teacher judgment or field test data 
to set passing scores. 




10 



20 



TABLE 5 



WHAT INFLUENCES THE DETERMINATION OF PASSING SCORES? 



inr luence 


Percent of districts 
responaing v n* J.U'f y 


Teacher judgment 


63% 


Community advisory 




input 


50 


Field test data 


72 


Other 


23 



Analysis of responses by demographic variables indicated that: teachers 
were less likely to be involved in the process in urban districts (45 percent) 
or unified districts (56 percent) than they were in other types of districts. 
Community members participated in about half of the districts, especially in 
middle and high income districts. Field test data were used in over two-thirds 
of the districts and were most often used by urban (80 percent) and suburban 
(76 percent) districts. 

About one-fourth of the responding districts report that information from 
other sources was used to determine passing scores. These included: boards 
of trustees (five percent); various types of committees, such as school site 
councils and proficiency task forces (five percent); statistical approaches 
(four percent); and other combinations of inputs (six percent). 

Passing Scores Specificity 

Eighty percent of the responding districts had a passing score for each 
content area. Most of the remaining districts had set passing scores for sub- 
tests within each content area. Only a few districts reported using a single 
score combining the three content areas (now proscribed by AB 801) or a passing 
score range (see Table 6). 



TABLE 6 

HOW SPECIFIC ARE PASSING SCORES? 



Percent of responding districts 





All 


Urban 


Suburban 


Rural 


Specificity 


(n"101) 


(n=I8) 


(n=48) 


(n-35) 


A combined passing score 










for reading, writing, 










and math 


3% 


0% 


6% 


6% 


A passing score for 










each subject area 


80 


72 


79 


83 


A passing score for each 










subtest within each 










subject 


30 


17 


31 


23 


A passing range score 


6 


11 


2 


9 


Other 


5 


17 


8 


6 



ERIC 



11 

3l 



Even when field test data are used, setting a single passing score is a 
somewhat arbitrary decision which needs to be revised and reviewed in light of 
studeflt performance and district remediation capabilities. A procedure for 
setting a "band-width" scoring range, which may be more fair to students, is 
discussed in Appendix M of the Technical Assistance Guide . 

Test Uses 

Proficiency test scores are used for determining student competency in the 
district's basic skills. Using test results for more than just certifying com- 
< petency is the beginning of integrating proficiency assessment into the regular 
program. 



TABLE 7 



HOW ARE TESTS USED? 





Percent of 




all responding 


Uses 


districts (n=103) 


Proficiency certification 


90% 


Assessing overall proficiency program 


34 


Course grading 


4 


Individual student diagnosis 


68 


Other 


7 



According to the findings of the 1979-80 study, proficiency test data were 
used similarly in most districts. Almost all responding districts used test 
data to certify proficiency in basic skills. Districts also reported using test 
data for diagnosis and remediation, evaluation of the proficiency program, and 
to a much lesser extent, in course grading. The districts also reported using 
curricular planning and improvement, course placement, staff development, and 
promotion/retention of students. 

Although each district established its own proficiency measures, some dis- 
tricts with common interests and geographical proximity had banded together to 
share proficiency tests. This occurred in one-third of the responding districts. 
Two patterns for sharing were evident from the data: 

1. Districts within a county had formed a formal or informal consortium 
for test development (20 percent of the districts). 

2. Districts maintaining high schools coordinate standards and tests with 
elementary feeder districts (13 percent). Smaller districts are more 
likely to share tests. 



Technical Assistance Guide for Proficiency Assessment (TAG) . Sacramento: 
California State Department of Education, 1978. 
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Proficiency tests were reviewed by Department staff and content specialists 
from school districts and universities to determine what skills were required of 
students and how those skills were tested. Each subject matter area was treated 
separately. Reading, writing, and computation content analyses follow. Other 
content areas tested by districts (such as social science) were excluded from 
review. Two specialists in functional transfer assessment reviewed test items 
I measuring life skills. 

Computation 

Almost all districts were requiring students to demonstrate proficiency in 
the four basic mathematical operations: addition, subtraction, multiplication, 
and division. Two-thirds of the districts included measurement on their tests. 
About half of the districts required computation involving percents; another 
half of the districts tested geometry (shapes, perimeter, area, and volume). 
Many districts assessed a student's ability to solve money problems and inter- 
pret graphs, tables, charts, or maps. Ten percent of the reading tests also 
included graphs, tables, charts, or maps (see Table 8). 

TABLE 8 



COMPUTATION SKILLS BEING ASSESSED 





Percent of districts 




assessing each skill 


Skill area 


(n=63) 


Addition, subtraction, multiplication, and division 




of whole numbers, fractions, and decimals 


95% 


Measurement 


67 


Geometry 


50 


Percentages 


50 


Money 


40 


Graphs, tables, charts, and maps 


40 


Consumer math problems 


40 


Time and wage problems 


25 


Ratios 


20 


Metrics 


10 


Averages, probabilities, and statistics 


5 



Districts frequently tested consumer math ability. About one-fourth of dis- 
trict tests had time and wage problems. A few tests measured metrics, ratios, 
probabilities, and statistics. 

Other features . The types of items included in the math tests were also 
reviewed. As expected, there was an overwhelming reliance on use of multiple- 
choice questions. A few districts employed a f ill-in-the-blank/completion 
format. 
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Three-fourths of the math tests examined did not measure math proficiency 
statements uniformly. There are many ways to teach and test similar concepts 
and operations in math. For example, an addition problem can be arranged hori- 
zontally or vertically , and it can have a plus sign or directions that indicate 
to the student that he or she is to add the figures. This is important, because 
the response mode on the test needs to parallel the response mode described in 
the proficiency statement. A related issue examined in the content analysis was 
the congruence between the proficiency statements and what was actually measured 
on the proficiency test. Ten percent of the districts listed proficiencies for 
which no items appeared on the tests. 

Curriculum specialists' review . Math curriculum specialists reviewed a 
subsample of 15 tests for content and test quality. 

Most of the 15 tests assessed "school skills"; a few required students to 
apply computational skills in life settings. An analysis of these life skills 
items appears in the Functional Transfer section, which follows. 

Although most of the math tests measured a common core of basic skills, 
curriculum specialists found tests that were especially easy (not testing 
fractions and decimals) and tests that were hard (assessing probabilities, 
ratios, and Roman numerals). 

With regard to overall test quality, one specialist remarked, "Given the 
constraints of local districts in developing these math tests, the tests were 
good measures of students' minimum levels of proficiency." 

Specialists were asked to make an informal judgment about the readability of 
the math test. Most math tests required a reading level of about eighth grade or 
less. Several tests required a great deal of reading within math items. Exactly 
what the items were measuring was ambiguous in these cases, because they required 
reading and problem solving abilities as well as math computation ability. 

Reading Comprehension 

District reading proficiency tests tended to be organized by reading com- 
prehension subskills or the context in which the skills were applied. All tests 
had reading passages, with test items drawn from those passages. Nearly one-half 
of district tests measured study skills and the ability to use reference materi- 
als. Many of the tests measured vocabulary and ability to follow directions (see 



Of those assessing reading comprehension subskills, about 25 percent mea- 
sured ability to find the main idea, to read for details, or to follow a se- 
quence. About 10 percent of the reading tests included maps, graphs, or forms. 

Proficiency tests tended to assess either academic reading ability or life 
skills content. Although all the tests that were examined had the multiple- 
choice item format, the stimulus material was quite varied. In passages assess- 
ing school skills, students were asked to read three or four paragraphs on 
school-related material. In life skills tests, stimuli included newspaper 
stories, advertisements, graphs, and illustrations taken directly from original 
sources . 



Table 9). 
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TABLE 9 



READING SKILLS BEING ASSESSED 









assessinc? each slcill 


Skill area 


(n»64) 


RoxiHino 1^ rtmm^oli one i o n 

I\C ClU ^ ^^UlLI L CilC ilO ^Llil 


1 00 

X V V 


$?t'iiHv sic 1 1 1 s / TP 'FpTPnf^p mAf'PTi ji 1 


45 


Vocabul ary 


40 


Following directions 


40 


Finding the main idea 


25 


Ppfldinc? ¥nr dptail 


25 


Following spoiipncps 


20 


Tjiteral/ inherent ial meanins 


15 




15 

X J 


Fact/ op in ion 


10 


Mflnfl And OTflnhic i irf oi^ma 1 1 nn 


10 

X V 


Q t* 1*11 ^11 1* A 1 An^lvaiQ 

Q I. L Ul» UU L CI X ClilClXjroXO 


10 

XV 




o 


Phonics 


5 


Decoding 


5 


Spelling 


5 


Signs and schedules 


5 


Newspaper articles 


5 


Letters 


5 


Abbreviations 


3 


Time 


3 


Reading rate 


3 



Ninety percent of the reading tests had items matched with district profi- 
ciency statements. A few districts listed proficiency statements that were not 
assessed. These included proficiencies requiring students to decode, understand 
the use of outlines, and demonstrate an understanding of phonics. 

The Fry readability formula was used to determine reading levels in cases 
where reading passages appeared difficult. In the six tests examined, the re- 
suits ranged from 6.6 to 8.4 in grade-level equivalents. These passages were 
characterized by complex sentence structure, difficult vocabulary, or use of 
jargon, but they were not, according to the Fry formula, overly difficult for 
hi^h school students. 

Writing 

A majority of districts required a direct measure or sample of student writ- 
ing. The number of required writing samples varied from one to eight per test. 
The student may have been asked to complete a form, write a paragraph, or compose 
an essay. Requirements included narrative, persuasive, receptive, and expository 
writing samples. Narrative writing samples called for the description of some- 
thing. Persuasive writing samples required an organized presentation of an argu- 
ment. Receptive writing samples asked for something, as in a business letter or 
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consumer complaint letter demanding satisfaction. Expository writing samples re- 
quired a description of a procedure or process. The districts called for each of 
the four types of writing in their proficiency tests in almost equal numbers. 

TABLE 10 



WRITING SKILLS BEING ASSESSED 





Percent of districts 




assessing each skill 


Skills assessed 


(n»61) 


Writing samples 


60 


Spelling 


50 


Punctuation 


45 


Form completion 


40 


Envelopes and letters 


35 


Sentence completion 


33 


Grammar 


25 


Usage 


25 


Mechanics 


20 


Paragraph analysis 


12 


Oral expression 


10 


Listening skills 


10 


Penmanship 


10 


Vocabulary 


8 


Use of outlines 


5 


Proofreading 


3 



Four methods were used to evaluate student writing samples: holistic, 
analytic, primary trait, and error count. Holistic scoring requires scorers 
to internalize a standard and score each sample as a whole. Analytic scoring 
requires the use of a scoring guide to focus attention on writing components of 
interest. Primary trait scoring evaluates those features of a writing sample 
that are relevant to the type of discourse. The error count is a frequency 
count of grammatical mistakes. Because less than 2Q of the responding dis- 
tricts included information about scoring methods used, it would be difficult 
to make accurate generalizations about the relative use of each scoring method 
in the field. 

The objective portion of districts' writing tests included spelling, punc- 
tuation, letter writing, and grammar. One-half of the tests assessed spell- 
ing; one*-third assessed letter writing and sentence completion; and one-fourth 
assessed grammar (usage and mechanics). Other content areas assessed under 
the rubric of writing included oral expression, listening skills, penmanship, 
vocabulary, outline use, and proofreading. 

For most writing tests, a direct congruence existed between the proficien- 
cy statements of the district and what was tested. In about 10 percent of the 
district testa j however, one or more of the proficiency statements were not 
assessed on the writing test. These tended to be oral expression or listening 
skill proficiencies. 




Curriculim specialists' reviey » The curriculum specialists noted that the 
writing tests required too much reading. The specialists suggested including 
more items measuring sentence manipulation; but they felt the items on punctu- 
ation and capitalization were well done. The most difficult items were those 
requiring students to arrange sentences according to sequence, to complete tax 
forms, and to interpret legal documents. The readability level of the writing 
tests was judged appropriate. The curriculum specialists recommended wider 
adoption of writing samples and the use of holistic scoring. 

Functional Transfer 

Functional transfer is a term for test items which measure school skills 
in an applied context. There is no legal mandate to include such items on 
proficiency tests, but many districts believe the intent of the law requires 
assessing the ability of students to survive in the world outside the school. 
Curriculum specialists reviewed a sample of district tests for functional trans- 
fer items, and they found that most districts had included some items as part of 
the reading and math tests. However, the specialists' judgment about the rele- 
vance of functional transfer items to student experience varied. Curriculum 
specialists recommended that districts review their tests to determine whether 
or not the items were appropriate for the students in their district. 

Functional transfer items were often used as alternatives to writing samples 
in districts where samples were not included. Functional transfer items required 
problem solving and included a wide variety of content, such as: (1) charts, 
maps, tables and graphs; (2) newspaper stories or articles; (3) signs and ads; 
(4) rulers; (5) business letters/addressing envelopes; (6) checkbook registers; 
(7) employment applications; (8) personal budgets; (9) price/cost comparisons; 
(10) medical or emergency directions; (11) bike/auto rules and regulations; and 
(12) time schedules. 



Test Characteristics 

In addition to information about test development and content, it is impor- 
tant to note what tests look like and how they are administered. The appearance, 
length, and directions affect student p^.rformance. 

Proficiency tests and accompanying materials were reviewed for: number of 
proficiency statements, use of item specifications, layout, test length, and 
test-taking instructions. 

Use of Proficiency Statements and Item Specifications 

The number of proficiencies assessed for all three content areas (reading, 
writing, and math) ranged from 12 to 150, with the majority of districts having 
assessed between 25 and 50 proficiencies. The more specific the proficiency 
statement was, the narrower the domain of items to be developed was. 

Over 90 percent of the materials had proficiencies that resembled behav^ 
ioral objectives, and only 10 percent had the detail of item specifications. 
The curricular implication of specificity is that more detailed diagnostic in- 
formation is available for the more specific statements. Only 35 percent of 



districts responding to the Measures Substudy included item specifications (many 
of these were test descriptions). Larger, urban districts were more likely to 
use detailed specifications* 

Layout 

The appearance of a test is very important in how a student feels and 
performs in a testing situation. In examining district tests, appearance was 
perhaps the most striking deficiency. Problems included small print, poor 
illustrations, and lack of space between items. Almost one-third of the tests 
had these layout flaws. Many district tests were poorly typed and reproduced. 
One test submitted was handwritten on a ditto master. 

On 20 percent of the tests, the items were crowded together, making the 
test very confusing. Students must have found it difficult to know where one 
item stopped and the next began. Anothei;/problem was small letters and numbers — 
some less than 1/16 inch tall (6 point type). Although more items could be 
placed on a page when the small type was used, this cost-saving measure seems 
hard to justify if the type limits readability. Another layout problem was the 
quality of illustrations. In many cases, pictorial and graphic items presented 
material critical to solving a test item. For such items, fuzzy or grainy 
illustrations can prevent students from answering the item correctly. These 
problems were exacerbated by ditto copying or xeroxing. 

This discussion of layout has focused on the worst third of the tests. 
About 25 percent looked professionally prepared, and another 40 percent of the 
tests were certainly adequate. It should also be noted that the tests submitted 
for this study were often field test versions that would have been revised again 
before used as a final assessment. Although layout is an important feature for 
any test, it is relatively easy to remedy with technical and grap<hlc assistance 
and resources. 

Test Length 

Test length is determined by the number of items and/or the time limits for 
administration. The number of test items (for the three content areas) in this 
study ranged from 52 to 392. Most tests averaged either 120 to 150 items or 200 
to 250 items. This bimodal distribution was due to the presence or absence of 
objective writing tests. Test length was obviously related to the time limit 
given to students for completing the test. Most districts did not report time 
limits. For those which did report on test length, there was a sizeable range 
(from three clasr. jieriods to about seven hours). 

Administrative Directions 

Instructions to students on how to take the proficiency test varied from no 
directions to relatively complete directions, including sample items and infor- 
mation on test-taking skills. About one-fourth of the proficiency tests had 
general directions to students about how to take the tests. One-fourth of the 
tests gave directions for each type of item but did not provide general instruc- 
tions. One-third gave general directions for item types and included sample 
items. 
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Very few of the directions in the proficiency tests included for the stu- 
dents a statement of the purpose and importance of the tests. This information j 
however, may have been given in the oral directions presented by the test admin- 
istrator* About 10 percent of the submitted tests provided test-taking clues to 
students, along with general instructions and sample items. These clues reminded 
students to check their answers and to erase stray marks, and the clues let them 
know that no penalty would be made for guessing. 

Cultural Bias 

Reviewing test items for cultural bias is important in minimizing differen- 
tial student performance based on sexual, cultural, or racial differences. The 
following information on reviews for test bias comes from the Implementation 
Substudy, described in Chapter 3. 

Only half of the responding districts in the Implementation Substudy re- 
ported examining their proficiency tests for cultural bias. When such proce- 
dures were used, districts reviewed the item content (45 percent), revised the 
tests on the basis of field test data (35 percent), and/or asked experts to 
review the tests for bias (22 percent). 

Districts that had not conducted test bias reviews cited the following rea- 
sons: the student population was essentially homogeneous; cotnmerical and con- 
sortium tests did not require bias reviews; test bias reviews are unnecessary; 
they had planned to, but hadn't yet; or they didn't know how. 



Chapter 3 

Implementation 

In the second part of this study, districts were asked questions about a 
variety of implementation activities. These will be reported as follows: 

!• Notice to parents 

2. Notice to, and involvemeat of, teachers 

3. Impact of proficiency assessment on curriculum and instruction 

4. Effects on students 
3. Conferences 

6. Remedial instruction 

7. Relationship with categorical programs 

8. Policies for special education students 

The information in this chapter is based on two data sources. The first 
data source is a summary of the responses to a questionnaire sent to a represen- 
tative sample of districts within California. The second source of data comes 
from 15 case study districts. Because this was not a representative sample of 
districts, the findings from the second source are not generalizable. 

Notice to Parents About Proficiency Assessment 

The Pupil Proficiency Law and notion^ of fairness towards students, as 
pointed out in the Debra P. v. Turlington decision, require that written notice 
of basic skills assessment be given to the student and to the student's parent 
or guardian. 

Questionnaire findings; In the Implementation Substudy, districts were 
asked when the information was reported to parents, how it was transmitted, and 
what information was transmitted. Tables 11, 12, and 13 give this information 
for districts maintaining high schools. 

Table 11 shows that only one-fourth of the responding districts notified 
parents prior to 1978, which was about one year after the first proficiency law 
was passed. Although it could take a year to develop proficiency standards and 
gain community input and consensus regarding those standards, the spectre of 
legal challenge on the grounds of due process makes the date of parent and stu- 
dent notification important. Notification of parents prior to 1976 could only 
have occurred in districts that were "ahead" of the law, that is to say, where 
proficiency assessment had already begun. 

Districts provided parents with information about grade levels to be tested 
and conference and remediation requirements as well as lists of proficiency 
statements. Several districts mentioned "other" information given to parents 
and teachers. These included: passing score information and district retention 
policies, as well as information that is not directly related to proficiency 
assessment, including Stull Bill requirements and district discipline policies. 



No. 78-892 (iv.-T-H CM/D. Fla. decision 7-12-79), 48 L. W. 2058C-24-79. 



30 



TABLE 11 



WHEN PARENTS WERE NOTIFIED 



Year parents were 


Percent of 


first notified 


responding 


about proficiency 


districts* 


testing 


(n-64) 


1971 


2Z 


1974 


2 


1976 


5 


1977 


15 


1978 


51 


1979 


25 



*High school and unified 



TABLE 12 



WHAT INFORMATION WAS PROVIDED TO PARENTS 





Percent of 




responding districts* 


Type of information 


(n-67) 


Copies of proficiency statements 


75Z 


Grade levels to be tested and when 


93 


Skill or competency description 




(item specification) 


49 


Conferencing procedures 


79 


Remediation requirements 


78 



*High school and unified 



TABLE 13 



HOW PARENTS WERE NOTIFIED 





Percent of 




responding 




districts* 


Means 


(n-67) 


Mail 


73% 


Meetings 


63 


Posted notice 


22 


School newspaper 


52 


Public newspaper 


84 


Individual conferences 


57 



*High school and unified 
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The data in Table 13 show that districts were using multiple means to no- 
tify parents. Twelve respondents indicated other ways of notifying parents, 
including letters, brochures, handbooks or newsletters sent home, Saturday and 
evening meetings, and radio and television spots. 

Case-Study findings : Case-Study districts were asked about parent notifi- 
cation procedures. Two aspects of the notification process deserve attention. 
First, while districts had taken elaborate steps to notify parents of the pro- 
ficiency requirements, there was no assurance that parents understood the impor- 
tance of such notices. One indication of this was the small number of parents 
attending informational meetings focusing on proficiency requirements. Comment- 
ing on notification, one superintendent said, "We have done just about every- 
thing, from informing here at the school to lots of events for the community. 
Kids know about the law, but the community just doesn't show up for these 
things." 

Second, cost was a heavy burden for districts trying to be conscien- 
tious about notification. As pointed out in interviews with staff, the cost 
to individual schools for notifying parents in large urban districts was exces- 
sive. Principals notified parents in advisory council meetings about tests, 
pass-fail rates, and remediation plans. Schools with high proportions of 
Spanish-speaking parents often translated the proficiency notices. Costs for 
translation, duplication, and mailing were met by individual schools. 

Notice to, and Involvement of, Teachers 

Teachers had a major influence on the integration of proficiency assessment 
into the curriculum and instructional program. 

Questionnaire findings : In the Implementation Substudy, districts were 
asked how and when teachers were notified of the proficiency requirements and 
what information was given to them. Table 14 reports the approximate date when 
teachers were notified. 

TABLE 14 



WHEN TEACHERS WERE NOTIFIED ABOUT PROFICIENCY ASSESSMENT 





Percent of districts responding 




High school/ 




Year of 


unified 


Elementary 


notification 


(n=67) 


(n=34) 


■1972 


2% 


0% 


1973 


2 


0 


1976 


6 


4 


1977 


18 


4 


1978 


58 


36 


1979 


15 


57 



22 



The data indicate that, like parents, teachers were notified primarily in 
1978 and 1979. Districts that notified teachers prior to 1976 were involved in 
basic skills testing before the proficiency laws were passed. It is not sur- 
prising that elementary districts notified staff in 1978 and 1979, as elementary 
districts were not required to implement proficiency assessment until 1978* 

The information reported to teachers and the methods of transmittal are re- 
ported in Tables 15 and 16. These data show that most districts gave teachers 
several kinds of information. As noted previously, the study did not investigate 
the quality or content of the information provided. For example, as indicated 
earlier in the Measures Substudy, districts used the term item specifications to 
mean many things. 



TABLE 15 



WHAT INFORMATION WAS PROVIDED TO TEACHERS 





Percent of 


districts 




responding 




High school 






and unified 


Elementary 


Type of information 


(n-67) 


(n-34) 


Copies of prof iciency/competency statements 


96Z 


97% 


Grade levels to be tested and when 


100 


91 


Skill or competency descriptions (item 






specifications) 


84 


77 


Conferencing procedures 


79 


68 


Remediation requirements 


88 


59 


Other 


10 


5 



TABLE 16 



HOW TEACHERS WERE NOTIFIED 





Percent of 




districts 




responding 


Mode of transmittal 


(n-100) 


Hail 


12% 


Meetings 


94 


Posted notice 


29 


School newspaper 


47 


Public newspaper 


54 


Individual conferences 


30 


Other 


13 



In almost all cases teachers were notified of proficiency assessment at 
meetings. Other means, such as newspapers and conferences, were used to notify 
teachers. 

Teacher knowledge of, and familiarity with, proficiency assessment is 
increased by involvement in test development. Areas of reported teacher 
involvement are indicated in Table 17. 



TABLE 17 



TEACHER INVOLVEMENT IN TEST DEVELOPMENT 





Percent 




of districts 




responding 


Type of involvement 


(n«104) 


Generating proficiency statements 


89Z 


Writing item specifications 


70 


Writing test items 


82 


Field testing 


88 


Setting passing scores 


83 


Other 


10 



Teachers were reported to have been involved in all aspects of test develop-* 
ment. Other areas of involvement cited were: working with outside consultants j 
selecting instruments, developing conference and recordkeeping procedures ^ 
developing remedial programs, and reviewing test results. 

Case-*Study findings ; The data for teacher involvement do not reflect the 
number or extent of teacher involvement in a given district activity. In one of 
the case studies, it was pointed out that while all teachers were given numerous 
opportunities to become involved at various stages of test development, most 
chose not to participate in any formal way. In another case^ that of a large 
high school district, math and English department leaders from each school did 
early test development work. The district office arranged for them to meet 
monthly, preparing item specifications first and then the items themselves. 



Impact of Proficiency Assessment on 
Curriculum and Instruction 

As an integrated part of the regular program, proficiency assessment has an 
impact on curriculum and instruction. 

Questionnaire findinga x- Survey respondents were asked to rate the impact 
of proficiency assessment on curriculum and instruction in their districts (re-* 
sponses are presented in Table 18). These responses reflect the perceptions of 
district-^level staff who might not have had direct knowledge of curriculum 
effects in schools. 
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TABLE 18 



IMPACT OF PROFICIENCY ASSESSMENT ON 
CURRICULUM AND INSTRUCTION 



Percent of districts responding 



Level of 
impac t 


All 
(n-99) 


Elementary 
(n-33) 


High school 
and unified 
(n-68) 


1* Significant 


33Z 


15Z 


42Z 


2. Marginal 


53 


46 


56 


3. No effect 


14 


39 


2 



High school and unified districts reported that prof iciency^iassessment had 
a ignificant effect on curriculum and instruction to a greater extent than it 
dici in elementary districts* This may have been the result of earlier implemen- 
tation requirements for secondary schools and the greater basic skills orienta- 
tion of elementary schools* Sise could also have been a factor in the degree of 
iiTfact* Larger high school and unified districts were more likely than small 
on«s to report a significant impact, possibly because proficiencies represent 
standardization of curriculum across many schools* Only 21 percent of the small 
high school and unified districts reported a significant impact, whereas 59 per- 
cent of high school and unified districts with populations of 10,000 or more 
reported significant impact* 

Respondents were asked to indicate what kinds of curricular and instruc- 
tional changes had occurred because of proficiency assessment* Their responses 
ara presented in Table 19* 



TABLE 19 

CURRICULAR AND INSTRUCTIONAL CHANGES 
REFLECTING PROFICIENCY REQUIREMENTS 



Percent of districts responding 







High school 


Unified 


Elementary 




Type of change 


(n-29) 


(n-40) 


(n-34) 


a. 


Revision of curriculuun materials 










by district staff 


69Z 


65% 


32% 


b.> 


Development of new materials by 










district staff 


57 


73 


29 


c 


Revision of curriculum materials 










by publisher 


7 


15 


3 


d. 


Reassignment of teaching staff 


52 


68 


3 


e. 


More time devoted to teaching 










basic skills or remediation 


93 


90 


44 


f. 


Modification of teaching methods 


55 


75 


38 


g* 


Addition of new courses 


69 


70 


18 


h. 


Other 


21 


15 


11 
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According Co district- level respondents, the most frequently cited effect 
on curriculum and instruction was more time being devoted to remediation and 
teaching basic skills (93 percent). Sixty-nine percent of the respondents indi- 
cated that new courses had been added and curriculum ma ^rials had been revised. 
New courses included "Math Essentials," "Consumer Math,' ind "Survival English." 
Additional program changes included activity centers and earning labs, changed 
sequence of instruction, more individualized instruction, dimmer school, smaller 
classes, and parent-as- tutor learning packets. 

Case-Study findings ; In several Case-Study districts, the cost of instruc- 
tion in basic skills came at the expense of an enriched curriculum and previously 
offered electives. A central office staff respondent in a large high school 
district questioned the trade-offs: 

We know that large numbers of kids don't write well, and if somebody 
told you, 'Make them write well,' what would it take? What the 
public is going to have to face is that you're turning the public 
schools into remedial factories. I don't quarrel with that as long 
as everybody understands that we have to give up other things to do 
that. Now our critics out there will say, 'Get back to basics where 
you should be; you're giving up frills.' But that word frill is 
something else to other people. It's not a frill to those parents 
who can't have their kids taught calculus anymore. 

A side effect of this orientation toward basic skills teaching and the re- 
assignment of staff is the negative effect on teacher morale of teaching primar- 
ily remedial courses. Several English and math teachers and department leaders 
indicated that teacher morale was low regarding proficiency assessment, because 
they were no longer teaching subjects which were challenging and interesting; 
rather, they were forced to teach "basic" subjects which properly belonged in 
the elementary grades. To the extent that high schools are forced to "do the 
elementary schools' job," teachers felt that the proficiency assessment effort 
was misplaced. 

Questionnaire findings : Respondents were asked what their districts were 
doing "to ensure that specific proficiency skills are included in classroom in- 
struction in the ways in which they are tested." (A term commonly used to de- 
scribe match between what is taught and what is tested is linkage.) Almost 
90 percent of the questionnaires included some description of linkage activi- 
ties. The responses were categorized into three types of effort mentioned: 
(I) provision of information to teachers; (2) involvement of teachers in the 
proficiency implementation process; and (3) review or modification of curriculum. 

Two-thirds of the districts ensure linkage by providing information to 
teachers; nearly one-half report that materials have been modified to reflect 
the proficiencies; and a smaller number actually involved teachers in various 
aspects of testing (see Table 20). 

The open-ended responses to the question on linkage varied widely in terms 
of the specificity and extensiveness of efforts cited; the following examples 
characterize this variation: 

Example 1 . Teachers have been given lists of skills tested and sample test 
items. 
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TABLE 20 



DISTRICT ACTIVITIES TO ENSURE MATCH 
BETWEEN TEST AND CURRICULUM AND INSTRUCTION 



Activities 



Percent of 
districts 



Percent of 
districts 

reporting this 
category as 

only activity 



CATEGORY I. Providing information in various 
forms to teachers 



66Z 



35% 



Proficiency statements 
Test descriptions 
Item specifications 
Sample items 
Practice items 

Student outcome data by item and area 
Scope and sequence listings 
Matrix of skills and instructional units 
Grading criteria (scoring) 
Materials for review: basic skills 
packets, remediation kits, etc. 
In-service opportunities to share and 
learn about proficiencies and remediation 



CATEGORY II. Involving teachers 



16 



Test development, selection, or revision 
Administering and scoring tests 
Developing scope and sequence guides 
Taking tests 

Using scoring criteria in instruction 
(rubrics) 

Reporting on skills covered in courses 



CATEGORY III. 



Teachers developing, modify- 
ing, or obtaining materials 



45 



17 



Curriculum guides, articulation guides 
Scope and sequence/matrix/continua of 
curriculum 

Course materials and instructional aids; 
practice tests, home use packets, etc. 
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Example 2 , Teachara compiled materials for testa~and received In-servico 
trainlng« 



ERIC 



Example 3 , Teachers helped develop tests and received copies of remedia- 
tion packets, manuals, and other descriptive materials. Also, 
the English curriculum has been revised for grades seven-ten. 
A scope and sequence has been developed in reading and math. 

Linking what is tested with what is taught may require greater effort for 
districts assessing functional transfer or "life skills" which have not tradi- 
tionally been part of the school curriculum. Respondents were asked in the Im- 
plementation Substudy to report on the relative emphasis of school skills and 
life skills in their proficiency test and curriculum. Thirty-nine percent of 
the responding districts reported assessing primarily school skills; only 15 per- 
cent reported assessing primarily life skills. A larger number (47 percent) 
reported assessing a combination of life and school skills. 

High school districts which reported a balance of "life skills" with "school 
skills" or primarily "life skills" were asked to assess the presence of these 
skills in their curricular/instructional program: 

1. Only one district reported little or no emphasis on "life skills" in 
the curriculum/instructional program. 

2. Thirty-five districts (67 percent) reported that the life skills they 
tested had always been in their curricular/instructional program. 

3. Sixteen districts (31 percent) reported that most of the life skills 
had been added to the curriculum/instructional program since the 
proficiencies were adopted. 

Integrating proficiency assessment into the regular pr -ram is a signifi- 
cant aspect of the implementation process but is difficult to describe without 
substantial school-level analysis, which was beyond the scope of this study. 

Effects on Students 

Four attempts were made in the current study to collect student performance 
data on the 1978-79 district proficiency tests. In the Measures Substudy, dis- 
tricts were asked to report the percent of the class of 1981 who passed the 
1978-79 test* In the Implementation Substudy, districts were asked to report 
the number of students failing the 1978-79 proficiency test and the nuniber of 
students receiving supplementary instruction. Central office staff interviewed 
in Case-Study districts were asked to report grade levels of test administration 
%n<^^ passing rates, by subject area for all students, special education students, 
aud racial and linguistic minority students. A final attempt to collect data 
consisted of a phone call follow-up to all 15 Case-Study districts. 

Separately or in total, the results of these four data collection efforts 
do not provide an accurate picture of pupil performance on the proficiency tests. 
The Department of Education's efforts were confounded by districts' reluctance to 
respond to its requests for such data and by their inability to provide complete 



data for student: popiuatlono and aubgroupa. Thu following data, In tha Do-^ 
partment*8 judgment > give the moat accurate portrayal available ot atudent 
performance* 

Meaaurea Subetudy findinga : Only 69 percent of the high achool and unified 
districta in the sample provided information on 1978-79 paaaing ratea. 



TABLE 21 



PERCENT OF STUDENTS IN CLASS OF 1981 
PASSING 78-79 PROFICIENCY TEST 





Percent of 




high school and 




unified districts 


Percent of class 


responding 


of 1981 passing 


(n-64) 


90-100% 


32 


80-89 


24 


70-79 


23 


60-69 


19 


Less than 60 


31 



The percent of districts failing large numbers of students makes Table 21 
appear quite alarming, especially when compared to the percent of expected ulti- 
mate failures taken from the 1979 study (an estimated 7.8 percent of the 1981 
seniors would not graduate). But other considerations mitigate against the in- 
terpretation that "too many" students are failing the proficiency test. First, 
the 1979 estimate may still be accurate; the 1978-79 testing was a preliminary 
assessment, the results of which may change in the second assessment. Second, 
the test instruments and/or the passing scores may be revised for subsequent 
assessments, given new information about students' real ability levels. Third, 
district remediation efforts are now underway that may assist students in pass- 
ing the part of the test they failed earlier. The paucity of data about percent 
passing and the fact that the data are based on an interim assessment combine to 
make these data less compelling for state level policymakers. Percent passing 
is a subject for within^district analysis, both by school and subpopulations. 

Implementation Substudy findings ; In the Implementation Substudy, dis- 
tricts were asked to report the numb^er of secondary students failing the 1978--79 
proficiency test, the number of students receiving supplementary instruction, 
and the percent of students in remedial programs who qualified for Title I or 
LES/NES services. 

While 67 districts maintaining high schools responded to the question 
(93 percent), the lack of a common recordkeeping system made the responses im- 
possible to interpret. Districts varied in how they kept pass-fail information, 
and whether or not performance records are maintained for population subgroups* 
Some districts reported scores by grade level and subject area for individual 
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population aubgroups in th« diatrict. Others reported only percent paaaing for 
the totftl population teated. Therefore, data £rom the Implementation Subatudy 
are not reportud here, aa it waa unreaaonable to create a aingle atandard of 
meaaure for auraraariasing the varloua waya in which diatcicta reported failure 
ratedt 

Caee-Study findinga ; In an effort to get more solid information on atudent 
performance, lettera and follow-up phone calls were made to the 15 Caae-Study 
districta. All 15 diatricta reported overall paaaing ratea that summarized the 
reaultB for their studenta. Th ree diatricta reported paas^fail rates by popula** 
tion subgroups (Tables 22, 23, 24, and 25) • The data in these tables do not 
represent generalizable patterns across the state. The mere fact that these 
districts collect such data make them anomalies. 

Administrators in several urban districts with large numbers of minority 
students expressed concern about differential student performance on district 
proficiency tests. Many districts were particularly concerned about having dis- 
proportionate numbers of minority students fail the tests. District administra- 
tors felt that it was too early to tell about effects on students; yet, when 
queried about minority students, these administrators acknowledged that minor- 
ity students were not likely to do as well on the test as the general student 
population. 

As part of site visits, one classroom of students in each school was given a 
questionnaire about student knowledge and experience with proficiency assessment 
(n«413). Since the classrooms were selected by school staff, this was a sample 
of convenience and is in no way a true cross section of California students. Re- 
sponses to the questionnaire indicate that almost all of the students knew they 
had to pass proficiency tests to graduate from high school (96 percent). Asked 
how they had been notified, students responded: 

^ A letter had been sent home (20 percent). 

- The principal or counselor made a school announcement (48 percent). 

- There was an assembly or school meeting (nine percent). 

• Other notification procedures had been used (15 percent). 

- Did not remember (21 percent). 

Students from Case-Study schools knew the following information about the 
proficiency tests : 

- What subjects were covered on the test (75 percent) 

- The kinds of questions that were on the test (33 percent) 

- The passing score (25 percent) 

- When the tests were given (18 percent) 

- Other information (three percent) 

- Were not given any information about the tests (15 percent) 

When asked how teachers had helped them prepare for the tests, students 
answered that: 

- Teachers talked about the kinds of things "asked on the test" 
(37 percent). 

- They reviewed basic skills (47 percent). 

- They talked about how to pass the proficiency Lasts (19 percent). 

- They "did other things" (three percent). 
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TABLE 22 



PUPIL PASSING RATES ON 1978-79 
PROFICIENCY TEST: CASE-STUDY DISTRICT A 







Percent 


passing 






Reading com- 










prehension 










(combined 








Ethnicity 


crades 9"* 








(enrollment. 900) 


10) 


Writing 


Computation 


Total test 


All students 


83Z 


20% 


52% 


11% 


White* 










Ethnic minorities: 










Black 


82 


13 


33 


0 


Hispanic 


73 


22 


42 


0 


American Indian* 










Asian* 










All others* 










LES/NES 


30 


0 


26 


0 


Special Education 










*Data were not provid 


led. 



TABLE 23 

PUPIL PASSING RATES ON 1978-79 
PROFICIENCY TEST: CASE-STUDY DISTRICT B 



Percent passing 



Ethnicity 
(K-12 enrollment, 
10 000) 


Reading com- 
prehension 
(combined) 
grades 9- 
10) 


Writing 


Computation 


Total test 


All students 


74% 


47% 


43% 


32% 


White* 










Ethnic minorities: 










Black 


48 


11 


14 


05 


Hispanic 


48 


23 


20 


15 


Anerican Indian* 










Asian 


90 


75 


60 


50 


All others 


50 


10 


10 


10 


LES/NES 


30 


10 


10 


5 


Special Education 


60 


40 


40 


30 



*Data were not provided 
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TABLE 24 



PUPIL PASSING RATES ON 1978-79 
PROFICIENCY TEST: CASE-STUDY DISTRICT C 





Percent passing, by subject 


and grade 


Ethnicity 


Reading 










(K-12 enrollment, 


comprehension 


Writing 


Computation 


37,000) 


Nine 


Ten 


Nine 


Ten 


Nine 


Ten 


All students 


75% 


81% 


61% 


70% 


29% 


41% 


White 


86 


90 


71 


80 


39 


53 


Ethnic minorities: 














Black 


61 


65 


49 


57 


13 


20 


Hispanic 


64 


73 


48 


56 


19 


29 


Asian 


76 


83 


70 


80 


45 


57 


American Indian 


70 


75 


49 


58 


18 


37 


All others 














LES/NES* 














Special Education* 















*Data were not provided. 



TABLE 25 



PUPIL PASSING ilATES ON 1978-79 
PROFICIENCY TEST: SAMPLE DISTRICT* 





Percent passing, by subject and grade 




Reading 










Ethnicity 


comprehension 


Writing 


Computation 


(enrollment, 4,000) 


Nine 


Ten 


Nine 


Ten 


Nine 


Ten 


All students 


71Z 


78% 


71% 


77% 


77% 


84% 


White** 














Ethnic minorities: 














Black 


54 


37 


57 


40 


34 


30 


Hispanic 


47 


56 


50 


50 


40 


50 


Asian** 














American Indian** 














All others** 














Other 


78 


85 


75 


85 


88 


95 


LES/NES** 














Special Education** 














*Table 23 is one more example oi 


: a distrj 


LCt's subgroup passing scores and is 



taken from a mailout request conducted by Stephen Rosenzweig, CRLA attorney. 
This table appears in an unpublished mimeographed report, ''California Profi- 
ciency Assessment: A Call for Reconsideration," p. 3. In conversation with 
Mr. Rosenzweig, it was learned that his mailout to more than 100 districts 
also generated very few responses. 

**Data were not provided. 
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Thirty-seven percent of the responding students had received no special 
help from their teachers to prepare them to take the test. 



Students knew what would happen to those who failed the proficiency tests* 
They reported their classmates who failed would: 

- Take the test again (60 percent). 

- Attend conferences to discuss test scores and next steps (14 percent). 
** Take a special course or do extra work (33 percent). 

- Did not know (24 percent). 

In the classes sampled, 88 percent of the eleventh graders had taken the 
proficiency tests (78 percent reading, 81 percent math, and 65 percent writing). 
Most students (75 percent) felt that the test was about the "right" degree of 
difficulty; the remainder (25 percent) was evenly divided between evaluating the 
test as "too hard" or "too easy." Of the 365 students who took the proficiency 
tests, 67 failed one or more parts. These 67 students responded to two ques- 
tions about remediation activities. Regarding their choice of "special work," 
two-thirds reported that they were assigned to a remediation activity, and one- 
third actually participated in choosing the remedial work. Students perceived 
that the remediation classes were beneficial, as 37 percent found them very use- 
ful, 48 percent thought they were somewhat useful, and 15 percent reported the 
remediation work was not useful. 

Questionnaire findings ; Particular concern was raised by the legislative 
subcommittee about the relationship between proficiency assessment and the stu- 
dent drop-out rate. Vfhile districts maintained records of nonreturning students 
and seniors who did not graduate, no data were available about student motiva-* 
tion for dropping out. District perceptions about the influence of proficiency 
assessment on. the drop-out rate were surveyed in the Implementation Substudy. 
Table 26 reports administrators' perceptions of the effect of proficiency 
assessment on the drop-out rate. 



TABLE 26 



ADMINISTRATORS' PERCEPTIONS OF EFFECT OF 
PROFICIENCY REQUIREMENTS ON DROP-OUT RATE 



Effect on 
drop-out rate 


Percent of 
all responding 
districts 
(n-87) 


Percent of districts respond- 
ing, by size of enrollment 


Low 


Medium 
(n-87) 


High 


Increase 


37Z 


25Z 


14% 


46Z 


No effect 


66 


71 


74 


46 


Decrease 


8 


4 


12 


8 



Most district administrators did not anticipate an increase in the drop-out 
rate as a result of proficiency assessment. Administrators in large urban dis- 
tricts anticipated a greater effect than those in smaller districts did. Caution 
must be taken in viewing these data, because Case-Study administrators reported 
difficulty in pinning down actual numbers and reasons for dropping out of school. 
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In summary, no comparative data were available on pupil performance on dis- 
trict proficiency tests. While districts maintained performance records, incon- 
gruities among local recordkeeping systems and political sensitivity prevented 
the Department from generating a single pass-fail rate. Case-Study reports sug- 
gested that minority students were not doing as well as majority students, but 
these findings were limited to cases. Further, inconsistencies across schools 
withiA districts suggested the need for more focused analysis of within-district 
variations. 



Conferences with Parents 

One of the provisions of the proficiency legislation requires a parent con- 
ference for those students who fail to meet the district adopted standards. This 
conference requirement raises the issue of timely notification to parents about 
their child's failure so that the remediation plans developed at the conference 
will immediately affect the student's instructional program. Another related 
problem is getting the parents to attend the conference once they receive notice 
of it. The law requires that the student's parent/guardian be requested in writ- 
ing to attend the conference. (The law further stipulates that if the parent or 
guardian does not respond, the school is required to make a "reasonable effort" 
to contact him or her by other means.) 

Questionnaire findings ; The Implementation Substudy attempted to determine 
when the test results had been communicated to the parents, and the substudy also 
posed questions about parent attendance at conferences. Districts were asked, 
"How long after administering proficiency tests are results communicated to par- 
ents of students who fail the proficiency test?" More than three-fourths of the 
districts maintaining high schools reported that test results were available to 
parents less than three months after the tests were administered. 

The districts were also asked in the Implementation Survey to report the 
percent of parents of tenth grade students who did not appear for conferences. 
Fifty-eight high school or unified districts (60 percent) responded to the ques- 
tion. Twenty-eight percent of these indicated that conferences had not been 
held. Nineteen percent reported that from none to one-fourth of the parents of 
tenth graders had not appeared. Fourteen percent of the districts reported that 
from one-quarter to one-half of the parents had not appeared. Another 14 percent 
reported that from one-half to three-fourths of the parents had not appeared. 
Finally, 11 percent of the districts reported that more than 75 percent of the 
parents had not appeared for conferences. Parent attendance in the 39 districts 
reporting varied widely, according to reports from district staff. 

Case-Study findings ; The purpose of the proficiency assessment conference 
is to design a remedial program for the student. In order to obtain information 
about district conference experiences, Case-Study districts were asked about 
their approaches and procedures to holding conferences; 

Use of individual conferences . In one large urban district, confer- 
ences at the high school were arranged and conducted by the counseling staff 
on an individual basis. Letters were sent home to the parents of the 80 stu- 
dents that had not passed. A follow-up letter was sent by registered mail 
informing parents of the need to set up an appointment. Individual confer- 
ences centered on the test results and the remediation options available. 
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Use of group conferences, with option for individual conferenc e. One 
school in a large urban unified district held both group and individual con- 
ferences. During the group conference, parents were informed of remediation 
opportunities for students and were told about the diagnostic information 
available from test publishers. Individual conferences were conducted by 
the counseling staff for students failing the proficiency test. In the in- 
dividual sessions, counselors covered the diagnostic information on the stu- 
dent's printout and designed an instructional program to help build student 
skills. 

Use of group conferences . Group conferences were set up in one small 
rural district for students who had not passed one or more of the profi- 
ciency tests. Parents of students who were not making sufficient progress 
were invited to an evening meeting. After a general overview of the law 
and the remedial program, the group was divided into three small groups by 
subject matter area in which the student failed. In the small groups a 
remedial teacher went over skill areas assessed, how the student would be 
reassessed, and what would happen if the student did not pass a second time. 

Arranging and conducting conferences were reported to be major logistical 
problems by several Case-Study districts. One counselor estimated that he had 
spent a week processing paperwork for proficiency assessment during which time 
he saw no students. Another counselor in a large high school in the same dis- 
trict who oversees the entire proficiency testing program said that he had spent 
six weeks of 60 hours per week on logistics and paperwork because the school 
has no computer facility. He entered 2,500 scores by hand and set up a special 
schedule of testing when the regular schedule did not permit a period long enough 
for test administration. The major reshuffling of schedules to administer tests 
and arrange conferences was and continues to be a problem in that school. 

In another district at a school where the population was predominantly 
Hispanic, conducting conferences was "not a major goal," according to the prin- 
cipal. The school sent one invitation to parents of students that had failed 
and was not planning any follow-up. The principal explained that this school 
had other priorities and would organize a comprehensive plan for conferences 
next year when he felt conferences would become more critical. However, another 
school in the same district with a very large black population made a vigorous 
attempt to bring parents in for conferences. The school had three documented 
mailouts of invitations, and between 15 and 20 percent of the parents had 
attended conferences. 



Remedial Instruction 

This study looked at district organization of supplementary or remedial 
instruction. As 1978-79 was the first year when proficiency assessment remedi- 
ation was provided, descriptive information was sought about who was responsible 
and what types of programs were provided. Structured interviews with principals, 
department heads, and remediation teachers in Case-Study districts raised other 
issues, including types of responsibility, addition of new courses, student 
entry, course content, and summer school. 

Questionnaire findings : Districts were asked in the Implementation Sub- 
study about primary responsibility for remediation programs, differences among 
schools' remediation programs, and the types of programs provided. 
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TABLE 27 



PRIMARY RESPONSIBILITY FOR PROFICIENCY 
ASSESSMENT REMEDIAL PROGRAMS 



Responsibilities 


Staff responsible, by percent 
of responding districts (n-93) 


Central office 
staff 


School-level 
staff 


Organization of classes 


12% 


68% 


Content of classes 


15 


85 


Selection of students 


9 


91 


Assignment of personnel 


20 


80 



One important and unexpected finding is indicated by the data in Table 27 • 
Although the law speaks of district responsibility in implementing a proficiency 
assessment program, it was found that school-level staff were primarily respon- 
sible for remediation programs in most reporting districts. Large high school 
and urxfied districts reported that the central office had responsibility for 
remediation programs more often than the central office did in elementary or 
smaller districts — except in the assignment of personnel* Small districts were 
more likely to rely on the central office to provide staff for remedial classes 
(29 percent of small districts versus 15 percent of large districts)* 

On the other hand, central office staff from 73 districts (68 percent) in- 
dicated that remedial programs were the same for all schools* Differences which 
were cited among schools included: learning centers versus individualized learn- 
ing programs, separate remedial classes versus remedial instruction embedded in 
regular classes, and differing availability of resources and tutors* The rea- 
sons cited for these differences were: local needs, varying failure rates, and 
influence of compensatory education programs* 

Case-Study findings : District allocation of responsibility to the school- 
level was also evident in the areas of recordkeeping, notifying parents, and 
conducting conferences with parents and students* Curriculum specialists in one 
large urban district participated in test development, provided teachers with in- 
service help on instructional linkage, and developed remediation packages based 
on the 7^12 curriculum continuum* These materials had standardized the content 
of remedial courses throughout the district* However, each school had its own 
plan for providing students with remedial assistance* In one school, the regular 
staff operated a learning lab for tutoring and remediation where students worked 
on remedial instruction materials developed by the central office staff* 

Questionnaire findings ; Almost all (96 percent) districts reported using 
regular classroom programs for remediation* Other types of remediation included 
special pull-out programs (59 percent), after school programs (25 percent), sum- 
mer school programs (41 percent), and tutorial programs (67 percent)* Fifteen 
percent of the districts reported using some type of remediation program other 
than the ones listed above, such as basic skills labs» independent study, Satur- 
day school 9 and Upward Bound* Elementary districts j large districts, and less 




affluent districts seemed more inclined to use pull-out programs. Elementary 
districts rarely used (17 perctint) after school or summer school programs for 
remediation due to proficiency testing. 

Case'^Study findings ; Case-Study districts reported using regular class- 
room programs, pull-out tutorial programs^ and suimner school for remediation. 
Several Case^Study districts reported that individual schools had multiple of- 
ferings for remediation. Courseo had been redesigned to emphasise the profi- 
ciencies as they were covered in the test. This was especially true in consumer 
math, composition, and sophomore English^i 

Several Case-Study districts reported creating new courses to meet profi- 
ciency remediation needs. In one large urban high school, 26 out of 46 math 
classes were remedial. Providing these courses has had a major impact on the 
school's elective courses. Regular classroom teachercT' had been reassigned to 
teach remediation. 

Most districts were not running highly centralized remediation programs but 
were instead leaving supplementary instruction primarily to individual schools. 
Responsibility for remediation devolved primarily upon building principals and 
teachers. Despite this, districts' coordinators per^ceived that programs were 
generally similar across schools within a given district. 



Relationship with Categorical Programs 

Categorical programs provide compensatory education for itocially and eco- 
nomically disadvantaged students and linguistic minority tttudents. Federal and 
state dollars are targeted to schools among eligible areas having high concentra- 
tions of low-income families. These programs include Title I/EDY and bilingual 
education. 

Remediation necessary for a student to meet proficiency assessment re- 
quirements is not specifically funded by categorical programs. However, like 
compensatory education, remediation programs to meet proficiency assessment re- 
quirements provide supplementary instruction in the bak^ic skills. And, it may 
be assumed, many students in need of remediation for proficiency assessment pur- 
poses are also served in compensatory education programs. The department of 
Education's data suggest that most districts are not keeping track of categori- 
cal program eligibility of students failing district proficiency tests. 

Questionnaire findings ; In the Implementation Substudy, districts were 
asked to report the percent of students in remediation programs who also quali- 
fied for Title I/EDY or LES/NES services. The fact that only 29 districts re- 
sponded and that their responses were incon&istent make these data i::ipossible to 
summarize and report. Districts were also asked to report on whether proficiency 
requirements had influenced bilingual programs. Approximately one-third of the 
92 responding districts that maintain high schools reported that proficiency 
assessment had influenced the bilingual instruction provided in their districts. 
These were large districts (a.d.a. of 10,000 or more), which tended to have 
higher percentages of limited-English-speaking students. Sixty-six percent of 
the responding districts reported no effect. 
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Case-Study findings ; Interviews in Case-Study districts revealed complex 
issues in the relationship between the remedial instruction needed by students 
to meet proficiency assessment and the categorical programs provided for students 
with special needs. 

Administrators in Case-Study districts were at once both reluctant to ac- 
knowledge the overlap of remedial instruction for proficiency assessment with 
categorical programs, and yet honest about how Title l/EDY funds were being used. 
Several district administrators felt that total overlap existed between remedia- 
tion for meeting proficiency assessment requirements and existing services fot* 
students with special needs. In one small rural district, Title l/EDY funds 
were being used in a remedial reading program, which existed side-by-side with 
"proficiency skills courses." 

In another district, the school with Title I funds had a variety of supple- 
mentary instruction programs, including special courses, small group settings, 
and labs with tutors. Another school in the same district without Title I funds 
used only large-group instruction within regular classes for remediation to meet 
proficiency assessment requirements. Still another district used Title I funds 
to hire a new teacher who was responsible for pull-out reading and mathematics 
instruction for proficiency assessment. 

Districts in the study were approaching the use of categorical funds for 
proficiency assessment cautiously. The low questionnaire response rate and 
Case-Study information suggest confusion about the interplay between state and 
federal laws. For example, districts were not sure whether they could use a Ti- 
tle I teacher in basic skill classes for students who had failed the district's 
proficiency test. 



The proficiency requirements do not exempt students in special education. 
However, district governing boards are given the option of providing students in 
special education programs with a "differential standard." Where provided, the 
differential standard is to be included in the student's lEP (individualized 
education program) . 

Questionnaire findings ; In the Implementation Substudy, districts were 
asked to report whether or not the local governing board had elected to provide 
differential standards for special education students. As reported in Table 28, 
most districts were providing differential standards for students in special 
education. 

Case-Study findings : Case-Study districts reported mixed approaches to pro- 
ficiency assessment of special education students. Several districts provided 
differential standards on an individual basis. In two other cases, special edu- 
cation students were given the opportunity to take the regular proficiency test. 
In these districts, if it was clear that the student would not be able to master 
the regular proficiencies, individual standards were written into the lEP. In 
yet another Case-Study district, the resource specialist was working exclusively 
on proficiency assessment activities with special education students. Case-Study 
districts operating under the state Haster Plan for Special Education reported 
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that the proficiency requirements and Master Plan programs facilitated the dif- 
ferential standards process. Special education staff raised the following con- 
cerns about the relationship between proficiency assessment and special educa- 
tion: (1) that the number of diagnostic assessments will rise sharply; (2) that 
placement in special education programs will be a loophole for nonprof icient 
students; and (3) that the current definitions for specific special education 
programs will become distorted. 



TABLE 28 



ADOPTION OF DIFFERENTIAL STANDARDS 



Board decision 


Percent of 
respond ing 
districts main- 
taining high 
schools (n«67) 


Differential standards 




provided (58Z): 




- On an individual basis 


46% 


- By disability category 


12 


Not provided 


13 


Not yet considered 


23 


Other 


6 




Chapter 4 

Case Studies 



Department of Education representatives visited 15 districts in the fall of 
1979 to gather more in-depth information about implementation efforts. A set of 
interviews was prepared and used for the third phase of this study. Interviewers 
then wrote detailed vignettes of individual site visits (see the methodology 
chapter for more information). Two of the vignettes, selected because of dif- 
ferences in the size and locale of the districts, are presented here as examples 
of districts' approaches to proficiency assessment. These two case studies are 
not intended to represent all districts in the state, but rather to document the 
different experiences and problems encountered by two very different districts. 



This unified school district of approximately 10,000 students is located in 
an urban area. The district has a 7.5 percent minority student population, the 
percent of aid to families with dependent children (AFDC) is mid-range, and there 
is a sizable LES/NES population and a 75 percent transiency rate. In the past 
three to four years, the district has undergone a number of important changes: 
integration, a new superintendent, and declining enrollment with its attendant 
effects of staff layoffs and teacher re&ssignments. 

On the whole, district personnel supported the proficiency law and were im- 
pleme;nting it conscientiously. The district was going beyond the statutory re- 
quirements for proficiency assessment in math, reading, and writing by adopting 
proficiency standards in social studies (and eventually in natural sciences). 
Along with the state proficiency requirements, the superintendent and board had 
mandated a districtwide competency-based education system covering the whole 
curriculum. Many district staff considered this additional mandate as too much 
too soon, and the additional requirements had resulted in strained relations be- 
tween the central office staff and the superintendent, as well as between the 
teachers and the superintendent. The central office staff found itself caught 
carrying out board policy, yet feeling little ownership of, or even philosophical 
agreement with, the district mandate. It was difficult, therefore, for those in- 
terviewed to focus on the state-mandated proficiency requirements independently 
of this otherwise stressful context. 



The director of curricular services was a central figure in the implementa- 
tion of the proficiency requirements—from test development to scoring, report- 
ing results, holding conferences, and notification. Early in the implementation 
process, he took a course in proficiency test development and worked with dis- 
trict committees to develop proficiency tests in each subject area. All schools 
were invited to participate on these district committees: in math, three schools 
were represented; in English, all five high schools were represented. Tests in 
math, reading, and writing were then passed on to the district curricular coor- 
dinating committees, which had teacher representatives from all schools. Though 
organized by the central office, there was plenty of opportunity for staff in- 
volvement, communication, feedback, test revision, and, most importantly, a real 
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sense of staff ownership. Those involved felt good about the process and felt 
proud of the results* Community and board reactions were that the standards of 
proficiency were too low« The director of curricular services responded that 
the community simply did not have a realistic idea of what the schools could and 
could not do. Schools ^ in his view^ simply do not have the sophistication to 
bring all students up to. higher levels of proficiencyi particularly in a district 
of this type. 

Notif ication, reporting of test results, and conferences were all handled 
from the central office. Testing was conducted in the spring. Last spring 
within a month of testing, letters were sent to all parents restating the re- 
quirements of the law and including a computer printout of students' performance 
in each subject area by subtest and by objective. In this letter, an invitation 
was extended to all parents to attend a meeting at their child's school to learn 
more about proficiency requirements and the remediation available for students 
who failed. At each school, approximately four families of failing students 
appeared for this meeting. 

Neither of the two principals interviewed had been involved in these steps 
before; however, both had attended the school group conferences. By and large, 
implementation of proficiency at the school level had been delegated to an assis- 
tant principal or the department heads. A large part of the work load at the 
school level was making sure students took the tests when they were offered and 
ensuring that those needing remediation got placed in appropriate classes. The 
director for curricular services felt that the process used in this district had 
successfully generated a feeling of staff ownership while at the same time pro- 
tecting the school (department heads and teachers) from the logistical burdens 
of scoring, reporting results, holding conferences, and the like. With the dis- 
trict office shouldering the major portion of implementation responsibility, it 
was hoped that the schools could concentrate on instruction. 

Curriculum and Instruction 

Proficiency standards in this district were derived largely from the curric- 
ulum. Even before the proficiency law had been signed, the district had begun a 
serious "back-to-basics" effort. Therefore, proficiency assessment was not an 
add-on, although it necessarily imposed an accelerated time frame. The math de- 
partment reported that, essentially, it had always been competency-based — the 
overlap or linkage between curriculum and proficiency standards being complete 
from the start. All those interviewed in the English department agreed that 
there had definitely been more attention given to specifying the curricular and 
instructional relationship to test content (linkage). The development of the 
district's writing packet for students and teachers was an exemplary product of 
the effort to establish linkage. All agreed, though, that establishing instruc- 
tional validity (determining if teachers were covering proficiency test content) 
was a major problem, which was likely to get worse as teachers were reassigned 
because of declining enrollment and increasing remediation needs. Administrators 
admitted that there was simply no way of ensuring that teachers were teaching 
what was being tested; however, they felt completely confident that curricular 
validity had been established (their curricular materials and guides included 
all proficiency standards). All teachers teaching basic math and English classes 
were encouraged to spend a week or so prior to proficiency testing reviewing the 
tests. All district categorical programs were focused on basic skills, most of 
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which could be found in the district proficiency tests. Hence, organization, 
funding, and content of categorical programs completely overlap the district's 
proficiency requirement: a* 



Remediation 

Remediatio.^ vas offered in a variety of ways: regular classroom instruc- 
tion, special classes during the regular school year, summer school classes, and 
a county drop-in center where students could follow an individualized program. 
At each school, all math and English teachers were informed about students in 
their classes who had failed proficiency tests and the areas of needed remedia-* 
tion. Teachers were then asked to focus instruction on these students, thus en-* 
couraging a tutorial environment. At all schools, basic math and reading classes 
were available to students below grade level and for students failing to meet 
certain levels of proficiency in basic subjects. These classes were concentrat- 
ing on "basic school skills." Summer classes, which were set up to help students 
gain the proficiency they needed, were offered and organized by the central of- 
fice. Out of 1,500 students who had failed some part of the proficiency tests 
in the prior year, 150 attended the 1979 summer session (100 in math, 50 in En- 
glish). The county drop-in arrangement offered an individualized way of obtain- 
ing help in gaining needed proficiencies. The program was new and the director 
of curricular services surmised that students had not taken advantage of the ser- 
vice yet. Remediation is judged "complete" when the student retakes and passes 
, . "v?st. Ihe county drop-in center provides retest opportunities monthly; the 
disitrict offers retest opportunities each spring. 

For now, remediation is not a major problem. But all persons interviewed 
anticipated a very big problem once parents and students faced the sanction of 
the students' not graduating in 1981. The district's high transiency rate was 
causing further concern — students simply are not in the district long enough to 
receive appropriate remediation. Teachers at both schools worry that as more 
remediation is needed in years to come, there will be a consequent deemphasis on 
enrichment and elective offerings. Remediation was expected to be useful, but 
it "isn't" going to provide the "miracle cure." Even with remediation, the di- 
rector of curricular services expects that 25 percent of the students will ulti- 
mately not pass the tests, and that these will be mostly ethnic minorities and 
LES/NES students. 

Results and Effects of Proficiency Assessment 

There was uniform concern among those interviewed about the pattern of pupil 
performance on proficiency tests. The district had set high standards. At the 
time of the study, 43 percent of the students had passed computation, 71 percent 
had passed reading comprehension, and 41 percent had passed writing. While these 
people were proud of the high standards set by the board, administrators and 
teachers were troubled by the disproportionate failure rates for ethnic minori- 
ties and LES/NES students in this district. The director of curricular services 
estimated that 10 percent of the blacks had passed, and less than 10 percent of 
the Hispanic group had passed. According to district administrators, a major 
problem for blacks was the strong local peer pressure not to participate in aca- 
demic school activities; hence, little motivation to demonstrate "proficiency." 
The problem for LES/NES students was considered to be a language problem. 
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Both principals and the director for curricular services indicated that 
the district had always been committed to providing help for special need stu- 
dents. All felt that there was total overlap between proficiency assessment and 
categorical programs for special need students. 

All mainstreamed special education students were being given the proficiency 
tests in the usual manner. If a student failed, the school team evaluated per- 
formance on the tests with the individual's student record to determine if an 
individualized test would be more appropriate. If a student failed again, al- 
ternative performance indicators were used. Finally, if special education stu- 
dents were judged to have tried "to the best of their ability," a diploma was 
awarded. 

The director of curricular services felt that the proficiency requirements 
would not hurt special education students. He was far more concerned about the 
borderline students who did not carry the "special education" label, but who 
would have great difficulty in passing the district tests. His hope was that 
this situation would eventually lead to a broadening of the definition of 
"special education" to include these obviously marginal students. 

There was uniform opinion among those interviewed that staff morale was not 
good. While most staff members supported the state proficiency mandate, staff 
morale was being severely tested by the superintendent's mandate for a district- 
wide competency-based system. To many, the hostility resulting from the basic 
philosophical conflict was swamping the potential positive effects of state- 
mandated proficiency requirements. One principal also noted that proficiency 
assessment, falling as it does at the end of a long line of state-mandated pro- 
grams, was not receiving the attention or commitment it deserved. The staff no 
longer has the energy to do the proficiency requirements justice. Some teachers 
commented on the likelihood of additional morale problems as more remediation was 
necessary and more teachers were assigned to basic skills instruction. Beyond 
this, teachers felt threatened by the district's plan to use student performance 
on proficiency tests to evaluate teaching. 

And yet, there was some feeling that proficiency assessment had, in fact, 
resulted in the development of a stronger and better sequenced English curriculum 
and curriculum guides. 

Student morale was perceived as being relatively neutral and "accepting," 
except among minority students. The administrative consensus was that as 1981 
nears, students were taking the tests more seriously. The director of curricular 
services expressed concern that the drop-out rate would increase as students re- 
alized they might not pass the tests. "If, in fact, proficiency testing is chas- 
ing kids away, a major purpose of the whole testing process is being defeated." 
On the other hand, one principal felt that attendance would increase. He felt 
that proficiency assessment would provide the incentive for students to overcome 
their skill deficiencies; hence, they would stay in school. 

Parent involvement in this district was low. District and school staff in- 
terviewed guessed that parents accepted proficiency requirements (even parents 
of failing sLudents). But all projected a cautious "wait and see" posture and 
felt a sense of uncertainty about parent reaction once June, 1981, arrived. 
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The "level of immediacy" was evident to administrators by the poor atten- 
dance at school conferences with parents in the prior spring. Parents had not 
realized the urgency of the situation* Despite numerous attempts to inform and 
generate community involvement, there continued to be little response. 

There was agreement that communication with the community needed more 
thought. Despite numerous attempts to inform and generate community involve- 
ment, there had been little response. Where the law was concerned, all felt the 
district and schools had done their part; but given the high transiency and high 
percentage of low income families, the district felt the need to find new ways 
to reach the community. 

Potentials and Problems 

The concept of "ownership" was important in the acceptance of proficiency 
assessment in this district. The local option feature of the law was considered 
essential to ensure relatively smooth implementation. Through district commit- 
tees on proficiency assessment, thereiwere opportunities to develop or borrow 
tests which reflected the standards of this community. From this came the writ- 
ing packet of which the district was exceedingly proud, the key, said the di- 
rector of curricular services and the English department heads at both schools, 
was the fact that the packet focused on instruction rather than the tests. It 
was felt by the staff that far more emphasis should be placed on classroom 
instruction — that the energies directed to the proficiency tests themselves were 
detracting from this more important and difficult area. As painful as the local 
test development process had sometimes been, the director of curricular services 
felt that to have been handed intact state tests would have been "disastrous." 

It was out of this feeling about the importance of "local" ownership that 
the director requested a different form of technical assistance than that pro- 
vided by the state to date. Rather than papers and workshops which came too late 
to be useful to the district, he would have liked to receive information about 
what was happening in other districts. He would have liked brief descrip- 
tions of effective and ineffective practices, with names and phone numbers of 
district contact people, so districts could share information with one another 
directly. Such a book could be updated on a regular basis as new ideas and 
practices were reported. 

Costs loomed central on the list of problems faced by the district. Dis- 
trict staff could not understand why funding was provided for notification, 
yet denied for initial test development, continuing item development, retest- 
ing, scoring, and reporting results. According to the director of curricular 
services, these were huge expenses and were "killing the district." 

This district had been conscientious in implementing the proficiency re- 
quirements. It had taken the task seriously and given considerable thought and 
time to the entire process. The staff had worked hard to develop a relatively 
good prof ic iency assessment system. But problems remained, and opportunity 
costs were becoming increasingly evident and serious as the time for withholding 
diplomas approached. 
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This high school district is rural, small (population 6,200) and ethnically 
homogeneous. Native Americans (4 percent) and Hispanics (7 percent) make up the 
only minority populations. Because of the size and character of the community, 
the district superintendent functions as both a civic and educational leader. 
He has been in the district six years and is involved in curricular and instruc- 
tional activities, as well as the administration of both the elementary and high 
school districts. 

Governance 

Implementation activities for proficiency assessment began in early 1977, 
when the superintendent traveled across the state to research the various 
approaches being taken by local districts toward proficiency assessment. 
The district management team developed standards with the aid of the faculty. 
Efforts were made to involve parents who typically did not actively support 
school activities because of their feelings that school staff were "better 
equipped" to make educational decisions. 

Teachers were very much involved in implementation, according to the admin- 
istration, and this involvement in test development sustained early staff morale 
and faculty commitment to the proficiency assessment process. Teachers were 
also given release time for developing reading and math test items. The focus 
of district proficiency tests was on consumer skills, such as those required to 
answer questions found on voter ballots, tax forms, and job applications. Fac- 
ulty from the high schools wrote the tests, and then the district contracted 
with a college group for psychometric review and revision of the measures. The 
college group turned in a disappointing performance, and the district was put in 
the position of administering its proficiency tests the first time with only 
minimal prior field testing and item examination. This lack of documented test 
validity or reliability was reported to parents, and instead of assigning passing 
scores, student performance was described in three broad categories: 

1. Students who performed so well that they were regarded as having 
passed (upper three stanines ) 

2. Students who performed in the lowest group, and are currently being 
offered supplementary instruction ( lowest three stanines) 

3. Students who performed at a level somewhere between the first two 
categories and who were retested on the revised district test 

Remediation 

Since valid test results were not available, the district had not begun 
formal conferences for proficiency assessment purposes at the time of the site 
visits. Students at one school who performed in the lowest third were advised 
in a group conference situation that supplementary instruction was available to 
them in the fall of 1979. After the January, 1980, administration of the test, 
individual conferences were to be held for all failing students. All students 
were sent one of five letters on proficiency assessment status during November, 
1979, outlining the general test results and district plans for a public meeting 
to discuss future administration of the test, conference options, and remediation 
alternatives. 
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Three instructional alternatives were available to students at the local 
high school: 

!• Remain with the regular course instructor, and focus instructional 
time on practicing the basic skills tested on the proficiency test* 

2. Agree to a contractual learning lab assignment, which was available 
to all students, as an alternative to the required courses of study. 

3* Agree to a pull-out remediation program assignment, which is credit- 
bearing and operates like a regular course. 

The district hired a new remedial English and math teacher with Title I 
funds* This person was responsible for teaching the third remediation alterna- 
tive* In this course, most of the instruction was individualized* 

The continuation high school had a pupil-teacher ratio of about 10:1 and 
instruction was individualized* A majority of the 75 students in the school 
were in need of remedial help in at least one of the basic skill areas, so re- 
mediation for proficiency assessment was built into all of the instructional 
programs* Since the first administration of the district proficiency test, 
23 students had dropped out of the regular high school and enrolled in the con- 
tinuation high school* The principal felt that proficiency assessment was 
directly responsible for the students* decision to transfer* 

Curriculum and Instruction 

All teachers were sent copies of the adopted proficiency standards and 
asked to identify where the proficiency skills were being taught in their 
courses* In some cases this meant major curricular revision in order that stu- 
dents be given experiences with applications of the basic skills in "real-life" 
contexts* There was a general feeling among faculty and administrators that 
"whether by design, or incidentally, classroom instruction will be based much 
more narrowly on test performance in the future*" 

Because the proficiencies were derived from the curriculum, and teachers 
were involved, the principals felt that the proficiencies either had been cov- 
ered in the curriculum prior to implementation of proficiency assessment or had 
been added/incorporated in the curriculum through new course materials* One 
principal felt that it was too soon to be sure about the effects of proficiency 
assessment on curriculum and instruction* He said, "Proficiency testing will 
change the focus in courses toward basic skills, but there is really no way that 
we can know exactly how unless we observe instruction* And I'm not sure we want 
to do that*" The district was in the process of revising the curriculum guides 
in early 1977, and it made the decision to postpone the revision process until 
the district could adopt proficiency standards and decide how the standards 
would be assessed* The revision of the curriculum guides was in process* 

District English and math courses had added specific emphasis on making 
an applied use of reading and mathematics, filling out job applications, demon- 
strating comprehension of voter instructions, and performing other applied skill 
activities • 



Results and Effects of Proficiency Aasessment 



The main effects of proficiency assessment in this district were on how 
teachers and administrators spent their work time and on curricular and in- 
structional emphasis within courses. Principals and teachers reported that con- 
siderable time had been spent in meetings during the past year and a half on 
de veloping standards, writing performance indicators, analyzing passing scores 
and scoring techniques, and administering tests. While the district had always 
had a heavy emphasis on the "three-Rs," the legal implications of proficiency 
assessment had compelled the district to approach test development in a more 
systematic fashion and to organize test administration, scoring, and reporting 
on a districtwide basis. 

Twenty-three percent of the high school population "failed" the 1978-79 
proficiency tests (approximately 270 students). Teachers, parents, and students 
had been notified of the proficiency requirements early in 1977 through general 
announcements about the law and graduation requirements. Notice was given at 
formal meetings, informal superintendent's coffee klatches, by newsletter, and 
by mail. Students were informed of proficiency requirements and the details of 
testing in a school assembly. 

There was a high level of awareness about proficiency requirements among 
students in the high schools. Only two of the 42 juniors surveyed were unaware 
of the law. There was some confusion about what the consequences of student 
failure would be, but students recognized that the tests were required for grad- 
uation, had been administered last year, and were about the "right level of 
difficulty," They were generally supportive of the legislative intent behind 
proficiency assessment but somewhat resented that "our class should be the ones 
to get stuck with the extra requirements," 

All special education students took the 1979 proficiency tests. Many of 
these students scored in the lowest three stanines, and the superintendent 
anticipated submitting a differential standards policy to the board sometime in 
early 1980, However, there was confusion and disagreement about appropriate 
policies for special education students, many people feeling that differential 
standards would be hard to justify. 

The district found a high degree of overlap in students who performed poorly 
on proficiency tests and those who scored in the lowest two quartiles on the test 
to qualify for Title I status. The same group of students needing proficiency 
remediation also qualified for the Title I program. 

Proficiency assessment had not had a significant effect on morale. The 
superintendent and one principal felt that it was too early to tell. The other 
principal indicated that proficiency assessment had had a negative effect on mo- 
rale. Several of the teachers interviewed seemed demoralized about the holding 
power of the proficiency requirements. Teachers were concerned that proficiency 
assessment seemed to be driving classroom instruction and would go away when a 
new idea came along. Most people felt that it was too early to know about the 
effects of proficiency assessment on student achievement nnd that there would be 
little or no effect on the drop-out rate. One principal felt different: 

Sure, proficiency assessment will affect drop-out rates. We 
are going to find a lot harder line. What makes us think. that 
kids can live with failure forever? We will see a lot of kids 
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dropping out • • • When they could gi^t a diploma for putting 
in a certain amount of time, it was one thing, but when they 
know they can't pass this test, they will just drop out. 

Teacher responses were mixed about the overall impact of proficiency assess* 
ment. While some felt that proficiency assessment would serve as a motivating 
force for students, to get students to take school seriously, others were skep- 
tical about the holding power of the law and the prospects of "teaching to the 
test." One said: 

I hope it lasts. It's needed. When you see people using the 
school district, I think we need some clout. When you put in 
extra hours, it needs to be worthwhile. Our department gener- 
ally feels that this is something we should be working on and 
are ready to put time in. 

While another observed: 

I think we are always skeptical of these things when they come 
out. I don't believe that every student can perform at a pre- 
scribed level. This law seems to stick them through a machine 
and expect them all to achieve the same ... I think we are 
dreaming if we expect all kids to achieve equally. 

Potential Benefits and Problems 

In appraising the district approach to implementing proficiency require- 
ments, the superintendent said that the most effective practices which the 
district had engaged in were: involving teachers in test development, which 
acted to have them internalize the process, and sending the principal and vice- 
principal to the state test development workshops, which were perceived as very 
useful in improving district confidence in their measures. The district was not 
very happy with its experience with the college group who had been contracted to 
assess the psychometric properties of the district test, and this district cau- 
tioned other districts to be careful in contracting with external agencies for 
local test development activities. 

The problems associated with implementing proficiency assessment in this 
district were typical of the problems of many small districts around the state. 

Articulating standards with elementary grades was a particular problem for 
districts maintaining high schools. The high school attendance area included 
more than the district's elementary feeder schools. Consequently, proficiency 
test requirements from the six elementary districts were not articulated with 
the high school standards. The district had gotten a limited amount of assis- 
tance from the county office, which helped several of the elementary feeder 
districts in adoption of similar measures. 

Both financial and personnel resources presented particular problems for 
the small rural district. The district did not have the resources to support a 
testing office, and the staff frequently juggled multiple assignments and roles. 
Testing expertise was limited. As mentioned, the district had an unfortunate 
experience with the group contracted to revise its tests. The test revision 



EKLC 



5R 48 



process thus fell upon the management team« While teachers were involved in 
the initial test development process, it seemed too time*-consuming and costly 
to bring them into the revision process. The result, however, was teacher dis- 
content with the new revision. "This doesn't look like what we wrote." In an 
effort to involve the teachers in the revision process and regain their support, 
the district was planning to bring in one of the state trainers who had worked 
with teachers before for a workshop on linking test results to curriculum and 
instruction. 

District teachers and administrators felt the following types of technical 
assistance would be useful: 

o A resource expert who could review district tests for psychometric 
qualities 

o Assistance on how to use test results in curriculum and instruction 

o Help in developing basic skills labs where activities would be cued to 
district proficiencies 

The district was experiencing problems in transmitting test information in 
a timely fashion to classroom teachers and students. Teachers were concerned 
about how to organize instruction for those students who needed remediation 
without sacrificing quality instruction for proficient students. 

Legislative requirements for school districts have a problematic effect on 
small districts. With limited personnel, districts like this one are compelled 
to give additional assignments beyond classroom instruction. The tasks associ- 
ated with implementing proficiency assessment require time and expertise which 
many small districts simply do not have. The district felt the need for resolu- 
tion of gray areas in the law — providing certificates of attendance, reciprocity 
of proficiency attainments among districts, legal requirements for remediation, 
and provisions for students in special education. 

This district was actively implementing proficiency assessment and trying 
hard to link proficiency assessment testing activities to curriculum and in- 
struction. Generally, school staff were in support of the proficiency require- 
ments, and interviews with teachers reflected underlying faculty support for 
holding students accountable for mastery of the basic skills. However, teachers 
felt that they had always focused on the basics. Proficiency assessment really 
"isn't new"; the only new part is the sanction of the diploma. Finally, people 
were generally in support of the local focus of proficiency implementation. The 
superintendent articulated this position in saying, "The validity of what we are 
doing is based on having local tests which are reflective of local curriculum." 

Proficiency assessment is making an impact. If the management problems can 
be worked out so that teachers receive test information on a timely basis and 
are provided with adequate resources for supplementary instruction (in-service 
training and materials), the district will meet the intent of the law and satisfy 
both legislators and local employers that students are being prepared "to deal 
effectively with adult life." 
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Chapter 5 

Technical Assistance Activities 



The Pupil Proficiency Law limits the role of the State Board of Education 
and the State Department of Education to providing technical assistance to local 
school districts. The Department is constrained from either developing a state- 
wide test or conducting monitoring and compliance reviews of district implementa- 
tion activities. The Department has a continuing program of technical assistance 
that is intended to be responsive to both the immediate and long-range needs of 
school districts. The following discussion reviews technical assistance provided 
to date and reports on Implementation Substudy findings on district technical 
assistance needs. 

The Department's Office of Program Evaluation and Research (OPER) has en- 
gaged in numerous activities to assist districts. In October, 1977, the Depart- 
ment released the Technical Assistance Guide for Proficiency Assessment (TAG), 
which is a process guide to implement the law. 

In response to the statutory requirement to provide "a range of assessment 
items" from which school districts might aeXect, OPER released in March, 1970, 
an item bank appropriate for grades seven-twelve et^ititled The Sample Assesamen t 
Exercises Manual (SAEM) . A second SAEM was released in February, 19797 for ele^- 
mentary grades. Both manuals include approximately 1,000 sample; test items and 
statewide field-test per;onnance statistics. 

Several appendices to the TAG h^ve been developed. These include: au ad- 
visory paper on legal issues, setting difi^rential st^^ndards, v^etectiug bi^-o 
in proficiency tests, and p,n analysis of the Florida test caso, D abra P. v. 
Turlington . 

In addition to thcpe advisory papers, the Department convened a task lorce 
to recommend approacht'is that might be effective in assistin^^ districts solve tbo 
problems of helping migrant stridents meet proficiency raquiremer ts. 

The Department has attempted to augment the guiles, iianuals, and advisory 
papers by training district administrators .and teachers in proficiency assessment 
measurement techniques. 

To further support this training and technicil assistance network, the De- 
partment han established a clearinghouse of effective proficiency 5^.s8C:3sment 
practices. The primary vehicle fct disseminating informatioa about effective 
practices is a quiirterly newsletter. Proficiency Tf.d^"?y» 

Various developmental activities are under nsid^ratiovi which may address 
some of the long-range issues raised by tlit^ lav. These include: instructional 
"packets" linking test tc?clts to curriculum and instruction, training in student 
root ivatic^ial techniques, and research in effective remediation practices. 

In the Implementation Substudy, district administrators were asked to assess 
the usefulness of: OPER's major technical assistance efforts, and their responses 
are reported in T^llti 29t 
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DISTRICT EVALUATION OF TECHNICAL ASSISTANCE 



Percent of all responding districts 







Somewhat 


Not 




Type of assistance 


Helpful 


helpful 


helpful 


Unknown 


Technical Assistance Guide (TAG) 


48% 


333: 


6% 


13% 


Sample Assessment Exercises 










Manual (SAEM) 


48 


32 


7 


13 


SAEM Workshops 


23 


33 


8 


37 


Proficiency Assessment Network 










Workshops 


13 


12 


4 


70 



The districts* evaluations of the Technical Assistance Guide (TAG ) and Sample 
Assessment Exercises Manual ( SAEM ) were very favorable. Both were judged helpful 
to about 50 percent of the responding districts and somewhat helpful to another 
one-third of the districts. 

Respondents were also given 16 options to identify their technical assis- 
tance needs. The most frequent topics cited as "very useful," in order of 
frequency, were: 

" Developing alternative modes of assessment 
" Motivating students who fail 

- Supplementary instruction 
Remedial strategies 

- Proficiency assessment of limited and non-English speakers 

- Using proficiency assessment results for diagnosis 

Those indicated as "somewhat useful" included, in order of frequency: 

- Detecting and eliminating test bias 

- Elementary-secondary articulation 

- Test construction and revision 

- Teaching test-taking strategies 

- Scoring writing samples 

- Reporting test results 

Those cited most frequently as "not useful," presumably because these tasks 
had been completed^ included field testing and setting passing scores. 
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Chapter 6 

Discussion and Analysis 



The major issues and problems raised by the study findings will be presented 
in this chapter, and some analysis of the implications of the findings will be 
provided* The discussion will be organized around three broad categories: 
assessment issues, instructional issues, and effects on students. 



Assessment Issues 

The Pupil Proficiency Law provides broad flexibility for local districts 
to develop or acquire their own tests. In light of this, the questions most 
frequently raised about district tests included: 

o How great is the variation in skills tested by districts, and are 
districts primarily testing "school skills" or "life skills"? 

o Is there wide variation in the difficulty of district tests? 

o Is the psychometric quality of district tests consistent? 

Each of these questions will be discussed in turn* 

How great is the variation in skills tested by districts, and are districts 
testing primarily "school skills" or "life skills"? Many questions and concerns 
have been raised about district definitions of "basic skills." Some people have 
speculated that districts with large numbers of high-achieving students might 
assess an inordinately high level of skills; conversely, some have feared that 
districts with primarily low-achieving students would assess unusually low-level 
skills. 

The Department's analysis of district tests does not support these concerns. 
The district tests which were reviewed reflected a relatively consistent view of 
basic skills across the required content areas of reading comprehension, writing, 
and computation. Very few districts had chosen to assess skills that were un- 
usually simple or extremely complex. In cases where districts had chosen to 
assess unusually low or high level skills, these skills comprised only a very 
small portion of the total test. 

The most interesting variation was in the choice of whether to assess 
"school skills," "life skills," or some combination of the two. According to 
the Department's questionnaire responses, 47 percent of those districts main- 
taining; high schools reported assessing a combination of equal numbers of both 
types of skills. Thirty-nine percent reported emphasizing primarily "school 
skills," while 13 percent reported emphasizing primarily "life skills." 

These findings have at least two important implications. Approximately 
one-third of districts testing primarily "life skills" reported that most of 
those skills were only recently added to their curriculum and instructional 
program. There is a significant question as to whether students in some dis- 
tricts have had an adequate opportunity to be taught the required skills. 
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Second, given the variation from district to district on testing of **schooI 
skills** or **Iife skills,*' a student transferring from a district that emphasized 
**life skills*' to a district that emphasized **school skills'* may not have an 
adequate opportunity to learn the skills measured by the new district *s test. 

Is there wide variation in the difficulty of district tests? This question 
is technically unanswerable. Each district is required to either design or ac- 
quire measures of student proficiency. Therefore, there is no common basis on 
which to compare either test difficulty or student performance from district to 
district. Only in cases where districts had adopted the same commerically de- 
veloped test or where the districts were drawing from a common item pool would 
it be possible to meaningfully compare passing levels. Less than 25 percent of 
the districts in the Department's statewide sample used commercial tests. The 
vast majority of districts used different tests and item pools. Therefore, it is 
not feasible to compare difficulty levels of tests. 

The Department informally examined test complexity using an expert review 
and content analysis. A group of curriculum content specialists reviewed the 
tests and test items to assess relative complexity. After they reviewed a sam- 
ple of computation and writing tests, the content specialists concluded that 
none of the tests was unusually complex or difficult, or beyond the levels en- 
visioned by the law as they understood it. The reading specialists felt it was 
inappropriate to make judgments about relative difficulty levels among tests. 

What is clear from the Department's analysis is that most districts were 
assessing the same types of basic skills. While the data revealed a few tests 
that were unusually easy or complex, on the whole the complexity of skills 
assessed on district proficiency tests did not vary to any great extent. 

Is the psychometric quality of district tests consistent, and how does it 
measure up to published psychometric standards ? Many factors affect the psycho- 
metric quality of district proficiency tests. Unless the district developing 
its own test has persons on its staff with adequate training in measurement 
skills or can hire such persons, it is conceivable that mistakes could be made 
in the test development process that would affect psychometric quality. By con- 
trast, districts buying a commercially developed test or selecting from a care- 
fully constructed item pool can be reasonably assured of having a test that has 
been constructed consistent with psychometric standards. There are advantages 
and disadvantages to each approach. 

The 1980 data revealed that if teachers in a district were involved in test 
development, selecting test objectives, writing or reviewing test specifications, 
or preparing test questions, the teachers felt a sense of commitment to, and 
ownership of, the test. The data suggested that district-developed tests were 
more likely to reflect local basic skills objectives and curricula and instruc- 
tional methods than were commercially developed tests. Case-Study interviews 
indicated that teachers often feel uncomfortable when working with a commercial 
test which they had no part in developing. They may, therefore, feel less *'own- 
ership*' of the skills tested and less responsibility for teaching those skills. 

There are definite trade-offs between locally developed and commercially 
developed tests. Commercially developed tests are likely to provide a sophis- 
ticated assessment package, with psychometric elegance. Locally developed tests 
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are likely to engender a greater sense of teacher ownership and more accurately 
reflect community needs and preferences. This, the Department believes, is a 
choice the law anticipated and one that districts are beat able to make. 



The three substudies indicate that district tests would benefit from care- 
ful review and refinement. Some of the weaknesses which were evident included: 
inadequate directions for test takers, poor layout, lack of item specifications, 
and failure to review for cultural bias. 

To be of maximum value, district-developed tests need to meet standards of 
practical utility as well as professional psychometric standards. Given train- 
ing, expert assistance, and time, the Department feels that the psychometric 
limitations in district-developed tests can be corrected. Development and re- 
finement of a test by district staff are continuous processes, which can result 
in a test which meets professional standards and also has practical value to 
teachers. 



Instructional Issues 

The pupil proficiency requirements are more than simply a "testing man- 
date." Development and ^administration of the proficiency test are only part of 
the broader requirements to provide basic instruction for all students in the 
skills measured by the test, notification and conferences for parents of stu- 
dents who fail periodic assessments, and remedial instruction for students who 
fail. 

Major questions raised about the instructional aspects of the law included: 

• Are the skills assessed by proficiency tests adequately reflected in 
districts' curriculum and classroom instruction? 

• Are requirements for basic skills assessment and instruction "taking 
over" the curriculum and leading to a deemphasis in areas such as 
social studies, science, art, and foreign languages? 

• Are parents of failing students responding to requests to appear for 
school conferences? 

• What kinds of problems are districts and schools having in conducting 
parent conferences? 

• How effectively have districts and schools been able to organize remedial 
programs for students who fail proficiency tests? 

Each of these questions will be discussed in turn. 

Are the skills assessed by proficiency tests adequately reflected in dis- 
tricts' curriculum and classroom instruction ? District administrators reported 
that proficiency requirements had had a demonstrable impact on curriculum and 
instruction, especially at the secondary level. Forty-two percent of the dis- 
tricts maintaining high schools reported that proficiency assessment had had 
a "significant" effect on curriculum and instruction; 36 percent reported at 
least a "marginal" impact on curriculum and instruction. In addition, more than 
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90 percent of high school and unified districts reported that more time was be- 
ing devoted to basic skills instruction than in the past; more than 65 percent 
reported revision of existing curriculum materials, development of new materials » 
and addition of new courses; and more than 60 percent reported reassignment of 
teaching staff and modification of teaching methods* Further^ virtually all 
districts (96 percent) reported providing teachers with copies of proficiency 
statements while somewhat fewer (84 percent) reported giving teachers copies of 
detailed test descriptions* 

These data, as well as Case-Study findings^ suggest that important efforts 
are being made to relate proficiency objectives to curriculum and classroom in-* 
struction* It should be noted that the present study made no effort to track 
the presence of proficiency-related objectives or instruction through district 
curriculum guides or systematic classroom observations* Even within a single 
district} this would be an enormous and costly taski which was beyond the scope 
of this study* 

Case-Study data suggest that in-service training of teachers was not widely 
used to orient teachers to proficiency requirements nor to train them in ap- 
proaches to teaching basic skills* While 80 percent of the districts respond- 
ing to the questionnaire reported teacher involvement in test development and 
scoring, the Department has no indication of the number of teachers actually in- 
volved* However, the Department feels that in-service training could be more 
widely used to assist teaching staffs in such tasks as using test results and 
providing remedial instruction* 

Are requirements for basic skills assessment and instruction "taking over" 
the curriculum and leading to a deemphasis in areas such as social studies, sci- 
ence, art, and foreign languages? This question is very difficult to answer 
given the scope of the study* Staff did not interview social studies, sci- 
ence, art, or foreign language teachers, nor systematically examine changes in 
required and elective courses offered by schools in the Case-Study sample* 

However, the Department did find that many more students than expected 
(30 to 50 percent) seemed to be failing at least one portion of the proficiency 
tests in grade ten* The Department suspects this has led to placement of many 
students in remedial classes and, presumably, lessened their opportunities to 
take higher-level courses* Undoubtedly, these students are losing the opportu- 
nity to be exposed to material in their regular enriched English and mathematics 
classes, or losing the opportunity to take advanced courses in these subjects* 
Whether they are limited in taking courses in other subject areas was difficult 
to tell at the time the study was conducted* 

In addition, information from the case studies suggests that the responsi- 
bility for teaching basic skills in Case-Study high schools is falling primarily 
on English and math instructors* When chairpersons of these departments were 
interviewed, they consistently reported that teachers in their departments were 
spending more time in basic skills and remedial instruction than they had in the 
past* In Case-Study districts, the availability of advanced courses in Engliish 
and mathematics was being limited in response to the emphasis on basic skills 
instruction* 

There is a curious paradox inherent in the question of whether basic skills 
are overwhelming the curriculum* Many students cannot demonstrate proficiency 
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in basic akilla. It Lh not clear how much they could bonuflt from more aclvancncl 
courscH, whether In l^ingLiBh, ma thematic a i or other subject areaot But, li; bCu*" 
dents are not challenged to aspire to learn higher-level content and Bkillsi 
there is i real danger that high school curricula could become trivialized* 
Certainly, this issue merits further study. Any additional examination ought i 
minimally I to examine such factors as direct student instructional time in basic 
Hkilla versus advanced or enrichment courses; changeH in numbers of basic skills 
verHua advanced or enrichment courses over time; perceptions of teachers of so- 
cial studies, science, art, and foreign languages; perceptions of students; and 
perceptions of parents* 

Are parents of failing^ students resj)onding to requests to appear for con- 
ferences? The requirement that schools hold conferences with the parents of 
students not making sufficient progress was intended to alert parents to their 
child's basic skills, strengths, and weaknesses; describe the remedial program 
the school would provide; and discuss ways in which the parent could help the 
child in basic skills at home. The law requires schools to notify parents in 
writing of their child's progress and the available date(s) for a conference. 
Should parents fail to respond to the initial notification, the school is 
required to follow up with another letter or with a telephone call* 

Data from the Implementation Substudy suggest that many parents of high 
school students are failing to appear for conferences. About 25 percent of 
the sample districts reported that less than half of the parents of failing 
students attended conferences at their schools. Eleven percent of the respond- 
ing districts reported that less than 25 percent of the parents appeared for 
conferences . 

Case-Study data suggest that districts were using a variety of methods to 
notify parents and schedule conferences* Several districts scheduled confer- 
ences in the evenings in an attempt to accommodate working parents. However, 
few if any of the approaches to notification or scheduling seemed to lead 
consistently to high rates of parent attendance. 

This low parent response may be due to several factors. First, the parents 
of high school students have not traditionally been as involved with the schools 
as have parents of elementary school students. To an extent, parents of high 
school students have not typically been asked to come to school in the past, and 
may be having difficulty adjusting to the new requirements* Second, despite ex- 
tensive efforts by districts and schools to notify them, many parents may not 
yet be aware of the seriousness of the ultimate sanction — denial of a high school 
diploma — attached to proficiency test failure. Many parents may view the profi- 
ciency failure notification as just another report of test scores or as a report 
card the school is sending home for their inf omation. 

With little more than one year left before diplomas are withheld because a 
student fails to meet the district standards, this problem of parent nonatten- 
dance at conferences seems extremely serious. The Department of Education is 
planning to develop an advisory paper for school districts which draws from the 
experiences of the Case-Study districts and suggests effective strategies in 
getting parents to attend conferences. Beyond this, it seems incumbent on dis- 
tricts and schools to make every possible effort to be sure that parents are 
aware of proficiency requirements and, if necessary, become involved in helping 
the school help their children master the required basic skills* 
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What kinda of problema are (litttricta an^^^^^ 
ent conferencee? Apart from llmitair parcint roapoua«, flistVictft ami achoola re- 
port two major problems with holding confarenceti: the logiatlcal prublema 
Involved In recordkeeping and achuduling conferenceu at timea whan appropriate 
ataff and parents can attend; and lack of funding to support staff (largely 
teachers and in some cases • counselors) to conduct conferences* 

The former concern (logistics) should ease as high schools gain more expe*^ 
rience with organizing and scheduling conferences. Funding to support staff 
coats for conferences may become available since the State Board of Control has 
voted to allow state reimbursement for conference expenses. What will be par- 
ticularly important, however, is exactly what level of reimbursement the Board 
of Control and the Legislature ultimately decide to provide. The level of reim- 
bursement should be adequate to permit high schools to conduct individual, rather 
than group, conferences for parents, and it must also allow for the costs of 
holding conferences in the evenings if this is necessary to ensure parent atten*- 
dance. There should also be some mechanism to ensure that funds provided for 
conferences are actually "passed through" by districts to schools. 

How effectively have districts and schools been able to organize remedial 
programs for students who fail proficiency tests? The Implementation Substudy 
asked who was responsible for remedial programs and what types of approaches 
were being used. For the most part, school districts were not organizing highly 
centralized remedial programs, but instead were leaving responsibility for reme- 
dial efforts to individual schools. In cases where schools have adequate re- 
sources to carry out the task and have teachers adequately trained to provide 
remediation, this may work well. But in instances where neither resources nor 
staff are adequate, the school may be left to piece together a program as best 
it can. For example, in one of the Case-Study high schools, a teacher who had 
in the past taught five periods of music was now assigned to teach one period of 
music and four periods of remedial math. It was also not clear how, with re- 
sponsibility for remedial teaching so widely dispersed among schools, sharing of 
effective approaches could be easily accomplished within a district. While many 
districts reported making standardized teaching materials keyed to proficiency 
tests available to schools, staff saw little evidence in Case-Study districts of 
efforts to design and share specific teaching techniques and strategies or to 
provide in-service training. 

The settings used to provide remedial instruction seemed familiar. Virtu- 
ally all districts were using in-class remediation, while about two-thirds re- 
ported using tutorial programs. About half the districts were using pull-out 
programs (special classes) and were planning to use summer school. In the case 
studies. Department staff observed that relatively few new settings for, or 
approaches to, remedial instruction had been developed at the secondary level. 
Many students in these programs still are hard to motivate, and it is some- 
times difficult to find teachers trained or willing to teach those basic skills 
courses. In light of this, some serious research on effective techniques of 
providing remedial programs for high school students seems warranted. 

Case-Study interviews surfaced two other issues. First, student absentee- 
ism emerged as a major problem, especially in the larger urban high schools. As 
one teacher in a large, urban high school in southern California commented: 

We've got good teachers and a good program here. Everyone is 
committed to teaching basic skills. But on any given day, 
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uta^^ily ona-fouvth ot our 4t;iui^ul;ii donU mUow up, m\A It'a h^nl 
to try t;o ItiAch baalc HkllU Co t^tuduutm who ar^uU h^rti* 



Whllo thia Htudy did not aynttunatlcMl ly axamLu^ tha ruUtlonahip o£ ntud^nl: ab-< 
aant^alam to pertornmnea on proticlency toata, it ie liki^ly that many chronically 
abaent atudenta ara alao por forming poorly on thaae testa • Thua, added to the 
raaponalbllitlea of many di*itricta and achoola to provide programa which will 
motivata atudenta to learn baaic akilla and to parauade parenCa of failing atu-^ 
denta to attend couferencea, there aeema to be a prior taak--«iotivating atudenta 
to attend achool in the firat place. 

Another isauo raised in the caae studies waa that o£ confuaion in high 
schools about the appropriateness of using categorical funding sources (for ex** 
ample I compensatory education and bilingual education funds) to help support 
remedial programs for failing students. Some high school principals addressed 
the issue cautiouslyi because they had apparently been given instructions that 
use of categorical funding for proficiency remediation constituted "supplanting'' 
of local and state funds. Other principals were less patient with the complex 
requirements and argued that in many cases the categorical funds were the only 
means they had to provide meaningful remedial programs. Schoolwide programs , 
such as school improvement , seemed to produce far less confusion i because these 
programs are explicitly designed to serve all students in a school rather than 
only an identified subgroup. In several Case-Study districts, school improve- 
ment was being used in a number of creative ways to improve organization of, and 
instruction in, basic skills programs. This study leads the Department to 
believe that schools may be suffering from a lack of clear direction as to the 
use o£ categorical funds for remedial programs for students who fail proficiency 
tests. State and district efforts to clarify these policies would seem helpful 
at this time. 



Effects on Students 

Ultimately, the weight of the pupil proficiency requirement falls on stu- 
dents. Under current law, all California seniors in the class of 1981, and in 
each class thereafter, will be required to meet course requirements and pass 
district proficiency tests in order to receive a high school diploma. Questions 
raised regarding the effects of proficiency testing requirements on students 
include: 

• Are students aware of proficiency testing requirements, and how are they 
responding? 

• How many students are failing proficiency tests? 

• Are proficiency testing requirements disproportionately affecting 
minority students? 

• How is proficiency testing likely to affect drop-out rates? 

• Are school districts adopting ''differential standards" for students in 
special education programs? 

Each of these questions will be addressed in turn. 
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Are students aware of proficiency requirements, and how are they responding? 
In each of the Case-Study schools, a small unrepresentative sample of eleventh 
grade students were surveyed in order to learn what they knew about proficiency 
requirements and their general reactions to the tests. Student awareness was 
encouraging. Almost all (96 percent) of the students surveyed knew that they 
had to pass proficiency tests in order to graduate from high school. Almost 
90 percent of the students responding had taken one or more proficiency tests, 
and three-quarters of these students felt that the tests were "about the right 
degree of difficulty." The remaining 25 percent was evenly divided as to whether 
the test was "too hard" or "too easy." Students who failed one or more of the 
tests were asked for their reactions to remedial programs. Eighty-five percent 
of the students found the remedial programs "very useful" or "somewhat useful," 
and only 15 percent found them "not useful." Thus, the students in the Depart- 
ment's limited Case-Study sample seemed to be aware of the proficiency tests, 
found them generally reasonable in terms of difficulty level, and seemed to be 
responding positively to remedial programs when these were provided. 

How many students are failing proficiency tests? In the Measures Survey 
questionnaire, districts were asked to report the percent of students in the 
high school graduating class of 1981 who had passed proficiency tests during 
the 1978-79 school year (as high school sophomores). About one"~half of the dis- 
tricts responding reported that 70 percent or more of their students had passed. 
Conversely, about half the districts reported that less than 70 percent of their 
students had passed. Of this latter group of districts, more than 30 percent 
reported that less than 60 percent of their tenth graders had passed. While 
districts were not asked to project failure rates for these students when they 
became high school seniors in 1981, the proportions of students failing as high 
school sophomores seemed high. Two caveats are important in considering these 
findings. First, despite follow-up requests, the response rate from districts 
on this question was relatively low (62 percent). Second, in some cases the 
passing rates reported may have been from "field tests" of district proficiency 
tests, rather than from administrations of the final tests the districts had 
decided on. 

Nevertheless, if 1978-79 failure rates are even close to the level the data 
suggest, several issues should be raised. First, it is possible that the ulti- 
mate proportion of students who would be denied diplomas in 1981 could be even 
higher than the 7.8 percent projected by district administrators and reported in 
the 1979 study. Second, with such high potential failure rates combined with 
some uncertainty about the ability of high schools to develop effective remedial 
programs quickly, there is some question as to whether the schools can, by 1981, 
adequately respond to needs of the large number of failing students. And final^ 
ly, as noted in the 1979 study report, it is not clear whef* ir local school 
district boards and communities will be willing to tolerat large numbers of 
students being denied diplomas in 1981. This could lead to pressure to reduce 
the difficulty levels of tests or lower passing scores. 

Reducing test difficulty or lowering passing scores may not, in some cases, 
be as heinous as it sounds. One of the central assumptions behind the Pupil 
Proficiency Law was that students having difficulty learning basic skills would 
be identified as early as possible and receive remedial programs that would help 
them learn the required skills in time to pass the proficiency tests before grad- 
uation. This a.'.^ sumption depended in large part on districts' ability to provide 
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efficient, sharply focused remedial programs that were effective in reducing 
failure rates. While it is certainly too early to draw any conclusions about 
the effectiveness of these programs, it may be that in some districts remedial 
programs are simply not working as effectively as the board or district admin- 
istrators would like. In such cases, boards may feel it is unfair to penalize 
students for program weaknesses, and they may feel inclined to reduce test 
difficulty or to lower passing scores. 

Are proficiency requirements disproportionately affecting minority students? 
Efforts to gather data on proficiency test failure rates for minority students 
were not very fruitful for a variety of reasons. First, districts are not re- 
quired to maintain records of test performance by racial or ethnic background of 
students. Therefore, mai.y districts do not even keep such records. Second, 
some districts which do maintain these records were unwilling to release them 
for fear that revealing the records might lead to or support legal challenges to 
their programs. 

From the very limited racial/ethnic data that were gathered from the case 
studies, it is clear that in the districts that provided data, black and His- 
panic students are failing proficiency tests at rates greater than those of 
white, Anglo students. For example, in one district, whites were passing the 
reading comprehension test at a rate of 86 percent, while blacks and Hispanics 
were passing at rates of 61 percent and 64 percent, respectively. In another 
district, all students wer?, passing the entire proficiency test at a rate of 
32 percent, while blacks were passing at a rate of 5 percent and Hispanics were 
passing at a rate of 13 percent. It is impossible to tell from the limited 
study data whether this pattern holds statewide. 

How are proficiency requirements likely to affect drop-out rates? It is 
virtually impossible to gather reliable information from districts on drop-out 
rates and to aggregate this information across districts. First, districts are 
not required to maintain information on dropouts. Second, simply knowing that 
a student has dropped out does not explain his or her motivation for leaving 
school. In order to find this out, the student would have to be traced and 
interviewed, a very costly procedure beyond the scope of the present study. 

However, both substudy and Case-Study questions solicited perceptions of 
administrators and teachers about dropouts. District administrators were asked 
in the Implementation Survey I ' ^mment on whether they felt the drop-out rate 
would increase, decrease, or ? ^ n the same. Thirty-seven percent reported 
they felt the drop-out rate wc increase, 55 percent reported that there would 
be no change, and 8 percent felt it would decrease. 

Some of the school staff interviewed pointed out that course requirements 
may have far more influence on a student's decision to drop out than proficiency 
tests would. This may be true, because as students progress through high school, 
they must pass enough courses to meet the district's "unit requirements" for 
graduation. Generally, a student cannot fail many courses and still accumulate 
enough "units" to graduate with his or her peers. If a student fails a large 
number of courses during the early years of high school, it is not likely that 
he or she will be able to make up credits during the senior year* Thus, whether 
or not the student has passed the proficiency tests, or feels that he or she 
could pass them during the senior year, the hopeless shortage of "units" may 
make the student feel it is not worth returning for the senior year. 




To attribute expected increases in drop-out rates to proficiency testing may 
be somewhat simplistic without considering the interaction of course and testing 
requirements for graduation* It is possible that since proficiency tests offer 
a relatively quick, focused means of meeting a portion of the graduation require- 
ments, these tests may be attractive to students who have seriously decided to 
commit themselves to studying for, and subsequently passing, the proficiency 
tests* 

Are school districts adopting "differential standards" for students in spe~ 
cial education programs? District boards are given the option to adopt policies 
providing for "differential standards" for students in special education programs 
who have diagnosed learning disabilities that would preclude them from attaining 
the district's regular standards, even with appropriate educational services and 
support. Should districts elect to adopt differential standards, the standards 
for each student must be included in that student's individualized education 
program (lEP) • 

The 1979 study reported that a relatively small number of districts (about 
one-*third) had adopted differential standards* At the time of the previous 
study, many districts had not yet considered the issue* Data from the current 
study are more encouraging* Of the responding high school and unified districts, 
58 percent had adopted policies providing for differential standards, 13 percent 
had elected not to adopt such policies, and 22 percent reported that they had not 
yet considered the issue* 

Of continuing concern is the relatively large number of districts that 
have not yet considered the differential standards option* By default, these 
districts have elected not to provide differential standards* 
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Appendixes 



Appendix I is documentation on the sample, and it includes definitions of 
the stratification variables and sample documentation. As discussed in the 
methodology chapter, the implementation sample was revised to generate a 
representative sample of high school and unified districts. 

Appendix II contains copies of instruments. Included are the Measures 
Survey and accompanying letter to district superintendents, which was sent 
on May 24, 1579, and the Implementation Survey, with accompanying letter to 
district superintendents, which was sent on October 16, 1979. 

Other instruments and documentation of procedures used in the 1980 study are 
available for examination at the Office of Program Evaluation and Research, Cali- 
fornia State Department of Education, 721 Capitol Mall, Sacramento. CA 95814. 
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Appendix I 

Sampling Framework 



For purposes of analysis, districts in the Measures Substudy were classified 
according to four dimensions , based upon data from the California Assessment Pro- 
gram files. These dimensions are type, size, AFDC, and geographic location* 
Each of these dimensions is described in detail below. 

District type ; Legislative time lines for elementary and high school dis- 
tricts are different. In order to reflect these differing time lines, districts 
were typed as either elementary, high school, unified elementary, or unified 
high school. 

District size ; Districts were classified as high, medium, or low, accord- 
ing to enrollment figures from the California Assessment Program files. The 
number of students associated with each of the classifications is listed below; 

Enrollment; 



AFDC ; The percent Aid to Families with Dependent Children (AFDC) was 
used as an index of district socioeconomic status. Districts were classified 
according to the following AFDC breakdowns; 

Percent AFDC; 



District location ; District location was classified as rural, urban, or 
suburban based on the following index of school location; 

Urban; City of more than 300,000 population 

Suburban: City of more than 100,000 but less than 300,000: 

1. Usually characterized as by itself, not as being near or part of a 
more populated area 

2. Located near a city of more than 300,000 

Community (incorporated as a city or town or an unincorporated area) of 
more than 25,000 but less than 100,000); 

1. Usually characterized as by itself, not as being near or part of a 
more populated area 

2. Located near a city of more than 300,000 



Large; 

Medium; 

Small; 



10,000 " 573,000 
1,000 - 9,999 
Less than 1,000 a.d.a. 



High; 

Medium; 

Low; 



12.20% 
6.30% - 12.19% 
0 - 6.29% 
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3. Located near fi city of more than lC0,00r but less than 300,000 

Community (incorporated as a ci'.:y or *'owz cr an unincorporated area) of 
2,500 " 25,000: 

1* Usually characterized as by itself, not as being near or part of a 
more populated area 

2. Located near a city of more than 300,000 

3. Located near a city of more than 100,000 but less than 300,000 

4* Located near a city or town of more than 25,000 but less than 100,000 

Rural: Rural area with less than 2,500 population: 

!• Usually characterized as by itself, not as being near or part of a 
more populated area 

2* Located near a city of more than 300,000 

3. Located near a city of more than 100,000 but less than 300,000 

4. Located near a city or town of more than 25,000 but less than 100,000 

5. Located near a city or town of more than 2,500 but less than 25,000 
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Revieed Sampling Framework 
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3# Located near a city of more than 100,000 but less than 300,000 

Community (incorporated as a city or town or an unincorporated area) of 
2,500 - 25,000: 

I* Usually characterized as by itself, not as being near or part of a 
more populated area 

2« Located near a city of more than 300,000 

3# Located near a city of more than 100,000 but less than 300,000 

4* Located near a city or town of more than 25,000 but less than 100,000 

Rural: Rural area with less than 2,500 population: 

1* Usually characterized as by itself, not as being near or part of a 
more populated area 

2# Located near a city of more than 300,000 

3. Located near a city of more than 100,000 but less than 300,000 

4* Located near a city or town of more than 25,000 but less than 100,000 

5* Located near a city or town of more than 2,500 but less than 25,000 
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Revised Sampling Framework 
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Appendix II 

Instruments Used in Study 
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8up«rint*nd«nt of Public Instruction 
cmd Director of Education 




STATE OF CALIFORNIA 

DEPARTOENT OF EDUCATION 

STATE EDUCATION BUILDING. 721 CAPITOL MALL. SACHAMENTO 958M 

May 24, 1979 



TO: District Superintendents 

FROM: Donald R. McKinley 

Chief Deputy Superintendent 

SUBJECT: Status of Proficiency Assessment Requirements: 
Survey of Measures 



In the 1978-79 school year, the proficiency provisions of the Education Code 
require elementary districts to adopt standards of proficiency in the basic 
skills and secondary districts to begin assessing pupils against district 
standards (Education Code Section 51215) . Any student who is not making 
sufficient progress toward attainment of the local standards is to be provided 
with a parent conference and supplementary instruction in the basic skills 
(Education Code Section 51216) . 

As you may recall, in August 1978, the Department conducted a voluntary survey 
of proficiency implementation. The survey provided the Legislature with back- 
ground information about the standards being adopted and initial implementa- 
tion activities. The Legislature was pleased to find in the analysis of this 
data a concerted, serious level of implementation in most districts. At the 
March 1979, budget hearings the Legislature requested an update on proficiency 
assessment implementation. In particular, the Legislature is concerned about 
the types of measures which districts are using and the nature of supplementary 
instruction/remediation programs in the basic skills. Although this survey 
is not on the Data Acquisition Calendar, it has been approved as an emergency 
survey by the Department's Data Acquisition Review Committee (DARC) . 

A twb-phaSpe study is planned to provide information about the content of 
district measures, the linkage between local measures and district curriculum/ 
instruction, and projected needs for supplementary instruction. Both legisla- 
tive and Department staff are interested in using this information in planning 
future technical assistance in the following areas: development/selection of 
diagnostic measures, alternative assessment, and approaches to supplementary 
instruction. 

- Phase I of this study is a content analysis of district measures. It is 
not an analysis of the validity or reliability indices of district tests. 
Rather, we are interested in the types of content being tested and the 
anticipated pass/fail rates based upon pilot test data or other district 
projections. 



67 

80 



Page 2 

District Superintendents 
May 24, 1979 

Phase II of the study, to be conducted In the fall, will examine the match 
between local measures and the district curriculum and Instruction and the 
provision of supplementary Instruction, which will focus on district needs 
for students who fall Interim assessments. 

Your district has been selected as part of a 10 percent random sample for 
this study. We are asking you to send copies of your proficiency measures 
and ancillary materials listed at the top of the survey to the Department. 
Data from districts will be held In confidence and used only for the purpose 
of this study. Individual district, responses will not be Identified. 

We are sending the survey at this late date, because It Is crucial that we 
Initiate Phase I of the study over the summer In order to anticipate needs 
for supplementary Instruction during the coming year. 

Recognizing the enormous demands made upon local personnel at this time of 
year, we have kept the survey Instrument as short as possible. It Is Impor- 
tant that the survey be completed by the person In your district who Is most 
knowledgeable regarding proficiency assessment Implementation. We recognize 
that elementary districts and elementary grades within unified districts may 
not have developed proficiency tests yet. Where this is true, please send us 
whatever materials you may have developed at this time. It is important that 
we receive these materials no later than July 1, 1979. 

Your cooperation in this survey is very important for legislative discussion 
and decision making. If you have any questions, please contact: 



Proficiency Survey Coordinator 

Office of Program Evaluation and Research 

California State Department of Education 

721 Capitol Mall, Fourth Floor 

Sacramento, CA 95814 

(916) 445-0297 
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Please return NO LATER THAN July 1, 1979 to: 



district label w/CDS code 



Proficiency Survey Coordinator 

Office of Program Evaluation and Research 

California State Department of Education 

721 Capitol Mall, Fourth Floor 

Sacramento, CA 95814 

(916) 445-0297 



To be completed by the person in each 
elementary, secondary, or unified dis- 
trict most knowledgeable about imple- 
mentation of Proficiency Assessment 

Please insert name and address of 
contact person: 



STATUS REPORT: PROFICIENCY ASSESSMENT MEASURES 



Checklist — Please check off the proficiency test materials for reading, writing, and math 
that you have enclosed: 

Proficiency statements 

Test(s) (all forms: label grades at which each is administered) 
Test descriptions (specifications) 
Summary of field test data 

Passing score information (cut point, procedures for establishing passing scores) 



INFORMATION ABOUT TESTS 
(Circle as many as apply.) 



1. Who developed the test(s)' 



a. District-developed by teachers 

b. District-developed by central 
office staff 

c. District-developed with assistance 
of outside consultant (s) 

d. Custom-developed commercially 
Specify firm 



Off-the-shelf commercial test 
Specify firm 



2. Are you using any items from item pools^ 

a. Yes, SAEM state-developed items 

b. Yes, public agency item pool 
Source 



c. ^Yes, private firm 

Source 

d. No. If not, where did you get your 
items? 



How were passing scores determined? 

a. Teacher judgment 

b« Community advisory input 

c. Field test data 

df Other (describe on reverse) 
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4. How are passing scores used? 

a. Combined cut score for reading, 
writing, and math 

b. A cut score for each subject area 

c. A cut score for each subtest within 
each subject area 

d. A passing"scdre range, rather than 
a single score 

e. Other — specify 



How are tests used? 



a. 
b. 

c. 
d. 
e. 



For proficiency certification 
For assessing overall proficiency 
program 

For grade in course 

For individual student diagnosis 

Other — specify 



6. Do other districts share this test? 

a. Yes (list cooperating districts on 
reverse) 

b. No 

7. Enter percent of this year's sophomore 
class (grade 10) who passed the profi- 
ciency test. 

% 
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Appendix Il-Continued 



WXLtOif BIU8 

Sttp«Tlni«nd«ii1 of Public InttructUm 
and Director oi Education 

1 




STATE OF CAUFORNIA 

DEPARTMENT OF EDUCATION 

STATE EDUCATION BUILDING. 721 CAPITOL MAa. SACRAMENTO 85614 

October 16, 1979 



TO: Selected District Superintendents 

FROM: Donald R. McKinley 

Chief Deputy Superintendent 

SUBJECT: Survey of Implementation of Proficiency Assessment 
Requirements 

On Hay 21, 1979, I sent a survey questionnaire which was part of a several 
phase study of proficiency assessment implementation requested in the 1979 
Budget Act. The phases of the study are: 

1* A content analysis of district measures, in progress, to describe to the 
Legislature the form and content of the district tests. 

2. A survey of the linkage and remediation process, which Is a follow-up 
on the initial 1978 data collection effort. 

3. A series of in-depth interviews in a sample of districts which will ex- 
amine the match between the content of local prpficiency measures and 
curriculum and instruction in the district. 
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Selected District Superintendents 
October 16, 1979 



As in our phase one study, data from districts will be held in confid nd 
used only for the purpose of this study • However, given the long^ter 
cations of the pupil proficiency law, the Legislature wishes to be kep p-to- 
date on implementation progress and technical assistance needs. 

Your cooperation is sincerely appreciated. ^se consult other staff wherever 

necessary in completing the questionnaire *der to make most timely use of 

the information, we will need to have youi. .;ted survey by November 1, 1979 • 
If you have any questions, please call: 



Proficiency Assessment Coordinator 
Office of Program Evaluation and Research 
721 Capitol Mall 
Sacramento, CA 9581A 
(916) 445-.0297 
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Enclosure 



Please return NO LATER THAN November 30, 1979, to 



Proficiency Survey Coordinator 

Office of Program Evaluation and Research 

California State Department of Education 

721 Capitol Mall, Fourth Floor 

Sacramento, CA 93814 

(916) 445-0297 



most knowledgeable about the 



To be completed by the 
person in each district 



implementation of Pupil 
Proficiency Assessment 



PUPIL PROFICIENCY ASSESSMENT SURVEY 



This questionnaire is designed to collect proficiency assessment data about 
the match between test content, local curriculum, and instruction. Please take 
about 15 to 20 minutes to respond to the questions that follow. Feel free to 
consult other district staff when necessary. Unified districts: please respond 
to the survey regarding implementation of proficiency assessment in the s^. condary 
grades. 

A. IMPACT ON CURRICULUM AND INSTRUCTION 

1. How would you assess the general impact of proficiency assessment on 
curriculum and instruction in your district? (Circle one number.) 

1. Has significantly affected curriculum and instruction 

2. Has marginally affected curriculum and instruction 

3. Has not affected curriculum and instruction (go to number 3) 

2. If curriculum and/or instruction have been revised to reflect profici- 
ency requirements, what chang, r nave been made? (Circle one number in 



each lettered row.) 



Yes 



No 



a. Revision of curriculum materials by district staff 



1 



2 



b. Development of new materials by district staff 



1 2 



What is your district doing to ensure that specific proficiency skills 
are included in classroom instruction in the ways which they are tested 
(e.g., teachers have been given copies of test descriptions). E>plain 
briefly. (Use page 11 if more space is needed.) 



Proficiency tests vary as to whether they assess "school skills" (e.g., 
multiplying decimals) or "life skills" (e.g., computing sales tax). 
Circle the option below which best describes your district's proficiency 
tests. (Circle one number.) 

1. Primarily "school skills" (Go to number 6.) 

2. Primarily "life skills" 

3. Equal amounts of each 

To what extent are the life skill proficiencies assessed in your test 
included in the curriculum/instructional program? (Circle one number.) 

1. We have little or no emphasis on "life skills" in the curriculum/ 
instructional program. 

2. Most of the "life skills" tested have always been in our curriculum/ 



7. Approximately when did the district first notify school staff about 
the content of proficiency tests? (Enter approximate dates.) 
Elementary: Secondary: 



month year month year 

8. What information were parents and teachers given about proficiency test- 
ing requirements? (Circle appropriate numbers in each lettered row.) 

To To To 

parents teachers neither 

a. Copies of prof iciency/competency statements 1^^ 2 3 

b. Grade levels to be tested and when 1 2 3 

c. Skill or competency descriptions (item 

specifications) 1 2 3 

d. Conferencing procedures 1 2 3 

e. Remediation requirements 1 2 3 

f. Other (explain) 



9* How were parents and teachers notified about proficiency testing 
requirements? (Circle appropriate numbers in each lettered row.) 

Parents Teachers Neither 

a. Mail 1 2_ 3 

b. Meetings 12 3 



How we^^ tochers involved in test development? (Circle one number 
In each lettered row.) 

Yes No 

a. Generating proficiency statements _1 2 

b. Writing item specifications (detailed descriptions 
of the skill being assessed and the manner of 

assessment) 1 2 

c. Writing test items _2 

d. Field testing 2 

e. Setting passing scores _1 2 

f. Other (explain) 1 2 



What test information is reported to terchers about individual 
students? (Circle one number in each lettered row.) 



Wot 

Yes No applicable 



a. Total test score (i.e., one combined test score) 1 



b. Subject area scores (e.g., math, reading, writing) 1 2 

c. Subtest scores within eac: subject: area (e.2, s 



How long after administering proficiency tests are results available 
to teachers ? (Circle one number.) 

1. Less than 1 month 

2. Between 1 and 3 months 

3. More than 3 months 

4. It varies (explain) 



How long after admini.stering proficiency tests are results communicated 

to parents of students who fail the proficiency test? (Circle one 
number. ) 

1. Less than 1 month 

2. Between 1 and 3 months 

3. More th^a ? I'o: \s 

4. It varies (eX4 .i^ain) 



AB 3408/AB 65 require that parents of students "not making sufficient 
progress" be invited to a conference. When are conferences held for 
students needing remediation? (Circle one number.) 

1. Less than 1 month 

2. Between 1 and 3 months 

3. More than 3 months 



SUPPLEMENTARY INSTRUCTION (REMEDIAL ACTIVITIES) 

> 

16. In generali are remedial programs for proficiency assessment the same 
for all schools in your district? (Circle one number.) 

1. Yes 

2. No 

If no, how are they different? 



3 7. Who has primary responsibility for remedial programs for proficiency 
assessment in your district? (Circle appropriate numbers in each 
lettered row.) 



Central School- 
office level 
staff staff 



a. Organization of classes 



1 



2 



b. Conent of classes 



1 



2 



c. Selection of students 



1 



2 



d. Assignment of personnel 



1 



2 



e. Other (explain) 



18. 



Do you use any of the iollovlng types of proficiency assessment remedial 



programs? (Circle one number in each lettered row.) 



19. FOR SECONDARY ONLY : VJhat teacher-pupil ratio is being used during 
1979-80 for proficiency assessment remedial programs? (Circle one 
number in each lettered row.) 

None Some Most 

of the of the of the 
programs programs programs 

a. Less than 1:3 1 2 3 

b. 1:4 to 1:15 1 2 3 

c. 1:16 to 1:30 1 2 3 

d. More than 1:30 12 3 



EFFECTS ON SPECIAL POPULATIONS 

20. How do you think proficiency requirements will affect the drop-out rate 
in your district? (Circle one number.) 

1. Increase 

2. No effect 

3. Decrease 

4. Other (explain) 



21. Have the proficiency requirements influenced bilingual instruction 
in your district? (Circle one number.) 

1. Yes 

2. No 



22. FOR SECONDARY ONLY : For each grade level below, please estimate ; 

(a) the number of students who failed the 1978-79 proficiency test; 

(b) the number of students who are receiving remedial instruction in 



1979-80 as a result of proficiency assessment; and (c) percent of those 



students 


; who also receive Title I/EDY 


or LES/NES bilingual services. 




Number of stu-- Number of stu- 
dents failing dents receiv- 


Percent (%) of students in 
proficiency assessment remedial 
programs who also qualify for: 


Grade 


1978-79 pro- ing remedial 
ficiency test instruction 


Title 11 

EDY funds LES/NES program 


7 






8 






9 


10 


11 


12 



23. The law provides that local governing boards have an option to provide 
"differential standards" for students who are 1 i special education 
programs and have learning disabilities which would preclude them from 
meeting the district's regular standards • Choose t^e option below which 
best describes the policy your board has adopted. (Circle one number.) 

1. District provides differential standards for students on an indi- 
vidual basis 

2. District provides differential standards by categories of student 



24. Has your district taken any steps to investigate, the possibility of 
cultural bias in your proficiency test? (Circle one number.) 

1. Yes 

2. No 

3. Do not know 

25. If yes, what procedures have you used? (Circle appropriate numbers.) 

1. Item content review 

2. Revision on the basis of field test information 

3. Expert /coiranunity review of test items 

4. Other (explain) 



26. If no, explain briefly. 



TECHNICAL ASSISTANCE NEEDS 

27. Would your district find technical assistance useful in the following 

areas? (Circle one number in each lettered row.) 

Not Somewhat Very 
useful useful useful 

a. Test construction and revision 1 2 3* 

b. Field testing 1 2 3 

c. Setting passing scores 1 2 3 

d. Detecting and eliminating test bias . . 1 2 3 

e. Developing alternative modes of 

assessment 1 2 3 

f. Scoring writing samples 1 2 3 

g. Data management/recordkeeping 1 2 3 

h. Reporting test results 1 2 3 

i. Conferencing with students and parents .1 2 3 

j. Using proficiency assessment results 

for diagnosis 1 2 3 

k. Motivating students who fail 1 2 3 

1. Supplement^ astruction/remedial 

strategies 1 2 3 

m. Teaching test« taking strategies .... 1 2 3 

n. Linking proficiency test content to 

curriculum and instruction . • 1 2 3 



28. Please rate the technical assistance materials and workshops provided 
by the State Department of Education. (Circle one number In each 
lettered row.) 

SomeT 

what Not Never saw 

Helpful helpful helpful or used this 

a. Technical Assistance 

Guide (TAG) 1 2 3 4 

b. Sample Assessment 
Exercises Manuals 

(SAEM) 1 2 3 4 

c. SAEM Workshops .... 1 2 3 4 

d. Proficiency Assess^ 
ment Networking 
Workshops (summer, 

1979) ........ 1 2 3 4 

29. Coats : Although districts are not required to keep detailed records 
of proficiency assessment costs, we are Interested In how much you 
have spent on Implementing proficiency assessment requirements. We 
would appreciate your attaching any documentation you might have 
about these costs. 



Thank you for taking the time to complete this questionnaire. If you have any 
additional comments about proficiency requirements, please add them in the 
space below or attach additional sheets as needed. 



Other Publications Available from the Department of Education 



Proficiency Assessment In California: a atus Report is one of approximately 450 publications that 
avaiJable from the California State Department of Education, Some of the more recent publications 
those most widely used are the following: 

Accounting Procedures for Student Organizations (1979) $ 1.50 

An Assessment of the Writing Performance of California High School Seniors (1977) 2.75 

Bicycle Rules of the Road in California (1977) 1.50 

California Guide to Parent Participation in Driver Education ( 1 978) 3. 1 5 

California Master Plan for Special Education (1974) l.OOt 

California Private School Directory (1980) 5.00 

California Public School Directory (1980) 1 i.oo 

California Public Schools Selected Statistics, 1977-78 (1 979) 1.00 

California School Accounting Manual (1978) 1,65 
California School Energy Concepts (1978) .85 
California School Lifting Design and Evaluation (1 978) .85 
California Schools Beyond Senano (1979) .85 
Child Care and Development Services: R^rpori of the Commission ormulate 

a State Plan (1978) 2.50 

Computers for Learning (1977) 1.25 

Discussion Guide for the C^alifornla School Improveinent Progi 1.50t* 

District Master Plan for School Improvement (1 979) 1.50* 

English Language Framework fox California Public Schools 19' 1.50 

Establishing School Site Councils: The California School Improve. ivut Program (1977) 1.50t* 

Evaluation Report of Consolidated Application Programs (1 979) 2.25 

Genetic Conditions: A Resource Book and Instructional Guide (1977) 1.30 

Guidance Services in Adult Education (1 979) 2.25 

Guide for Multicultural Education: Content and Contex' { 9. i 1.25 

Guide forOngoing Planning (1977) 1.10 

Handbook for Instruction on Agitig (1978) 1.75 

Handbook for Planning an Effective Reading Program (i <^ ) 1.50* 

Handbook for Reporting and Using Test Results (1976) 8.50 

A Handbook Regarding the Privacy and Disclosure of Pupil Records (1 978) .85 

Health Instruction Framework for California Public Schools (1 978) 1.35 

Improving the Human Environment of Schools (1 979) 2.50 

Liability Insurance in California Public Schools (1 978) 2.00 

A New Era In Special Education: California's Master Plan in Action ( 1 980) 2!oO 

Parents Can Be Partners (1978) 1.35t 

Pedestrian Rules of the Road in California (1979) 1.50 

Pedestrian Rules of the Road in California, Primary Edition (1980) 1.50 

Physical Education for Children, Ages Four Through Nine ( 1 978) 2.50 

Planning for Multicultural Education as a Part of School Improvement (1 979) 1.25* 

Planning Handbook (1978) 1.50* 

Publicizing Adult Education Programs ( 1 978) 2^00 

Putting It Together with Parents (1 979) [85t 

Resource Catalog for Proficiency Assessment (1978) .85 

Sample Assessment Exercises, Proficiency Assessment, Elementary (2 Vol.) (1978) 54.00 

Sample Assessment Exercises, Proficiency Assessment, Secondary (2 Vol.) (1 978) 54.00 

Science Framework for California Public Schools (1978) 1.65 

Site Management (1977) 1^50 

Social Sciences Education Framework for California Public Schools (1975) 1.10 

State Guidelines for School Athletic Programs (1978) 2.20 

Student Achievement in California Schools (1979) 1.25 

Students' Rights and Re^onsibilities Handbook (1 980) l.'50t 

Survey of Basic Skills: Grade Three- Rationale and Content (1980) 1.50 



